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This  paper  reports  the  results  of  investigations  undertaken  in  a continuii 
program  to  develop  decision  aids  for  Navy  task  force  commanders. 

The  focus  in  this  contract  phase  was  on  the  contribution  of  advanced 
techniques  of  decision  analysis  to  action  selection  situations  in 
the  mission  execution  phase  of  task  force  operations.  In  particular, 
the  in  sstigatlons  centered  on  testing  and  refining  the  prototype 
interactive  computer  graphic  aid  that  was  identified  under  a previous 
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A pilot  tost  of  this  aid  vaa  conducted  in  order  to  eatablleh  the 
conceptual  acceptability  of  the  aid  to  operational  Navy  eubjects  and  to 
identify  areaa  for  aid  refinement.  In  support  of  the  test,  the  anti- 
ship  missile  (ASM)  threat  was  examined  and  modeled. 

Additional  investigations  were  made  into  methods  of  enhancing  the 
aid  to  consider  more  state  hypotheses,  to  Incorporate  a model  of  time, 
to  address  a decision  maker's  attitude  toward  risk,  and  to  establish  a 
more  generalized  use  of  the  aid.  An  initial  attempt  was  also  made  to 
establish  matching  principles  between  the  aid  and  decision  situations 
existing  in  the  naval  tactical  environment.  a 


As  a result  of  the  pilot  test,  several  modifications  and  refine- 
ments were  made  to  the  prototype  computer  software,  and  a user's  manual 
is  being  prepared  for  support  of  testbed  implementation. 
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EXECUTIVE  SUMMARY 


Introduction 


This  Technical  Report  summarizes  the  results  of  the 
third  phase  (FY  76)  of  research  undertaken  in  support  of  the 
Office  of  Naval  Research  Operational  Decision  Aids  (ODA) 
project  by  Decisions  and  Designs,  Inc.  (DDI)  of  McLean, 
Virginia.  The  report  represents  eight  months  of  research 
effort  in  a continuing  program  in  which  various  technolo- 
gies, including  modern  decision  analysis,  are  used  to  ex- 
plore the  Navy's  tactical  decision  environment  for  potential 
applications  of  their  respective  methodologies  in  the  form 
of  computer-graphic  decision  aids.  The  goal  of  this  research 
program  is  the  eventual  specification  of  a decision  aiding 
sub-system  as  an  integral  part  of  a higher-order  tactical 
command  and  control  system  to  support  naval  task  force 
commanders  engaged  in  crisis  and  combat  operations. 


The  objective  of  the  DDI  effort  has  been  to  identify, 
combine  and  implement,  in  a computer-graphic  form,  the 
various  decision-theoretic  properties  that  would  be  appro- 
priate to  a class  of  decisions  found  in  a tactical  commander's 
contingency  planning  and  execution  phase  of  naval  operations. 
Research  and  investigation  to  date  has  revealed  several 
areas  of  apparent  methodological  application,  and,  within  a 
tactical  user  sense,  a number  of  promising  decision  aiding 
procedures  have  been  identified  for  further  development  and 
evaluation. 


A compact  set  of  prototype  aid  specifications  has  been 
developed  for  application  and  evaluation  in  a workshop  and 
testbed  environment.  DDI  is  presently  refining  the  initial 
software  for  an  interactive  computer-graphic  decision  aid 
(Execution  Aid)  designed  to  assist  a naval  commander  in 
planning  for  coping  with  the  anti-ship  missile  (ASM)  threat 
during  the  execution  phase  of  task  force  operations.  Addi- 
tionally, a preliminary  specification  for  assisting  a 
commander  in  the  offensive  mode  of  battle  management  has 
been  designed  for  computer-graphic  implementation  and  pilot 
testing  during  a follow-on  phase  of  research.  Investigation 
is  also  continuing  in  an  effort  to  find  a more  generalized 
use  for  the  various  decision  analytic  properties  currently 
embodied  in  the  prototype  aids  under  development  and  evaluation 
as  described  above. 


Research  Tasks 


The  research  and  investigatory  effort  consisted  of  the 
following  tasks: 
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o The  development  of  evaluative  criteria  for  deter- 
mination of  the  conceptual  value  of  the  prototype 
Execution  Aid  as  perceived  by  operationally  ex- 
perienced naval  personnel  in  pilot  testing  trials. 

o Pilot  testing  in  the  form  of  structured  workshop 
trials  in  which  a decision  analyst  and  a qualified 
naval  subject  exercised  the  prototype  Execution 
Aid  in  a simulated  threat  situation  utilizing  an 
interactive  computer-graphic  terminal. 

o Modification  and  refinement  of  prototype  aid  soft- 
ware to  incorporate  recommended  additions  and 
features  elicited  from  the  pilot  testing  procedure. 

o Preparation  of  a user's  manual  to  support  adaptation 
and  implementation  of  the  prototype  aid  in  various 
testbed  environments. 

o The  development  of  a Bayesian  hierarchical  inference 
model  to  exhibit  the  Soviet  anti-ship  missile 
(ASM)  threat  to  form  the  basis  for  aid  config- 
uration. 

o Investigation  of  decision  aid  generalization 

for  the  adaptation  of  potential  aids  to  a broad 
class  of  tactical  decision  situations. 

Research  Activity 

Framework  for  prototype  execution  aid  trials  - Although 
preliminary  in  nature,  the  design  for  the  pilot  test  of  the 
prototype  Aid  required  a systematic,  comprehensive  and 
convenient  form  for  identifying  the  strengths  and  weaknesses 
of  the  Execution  Aid  as  evidenced  by  the  test  subject  responses 
to  Aid  trials.  A multi-attributed  criteria  matrix  for  Aid 
evaluation  was  investigated  and  discarded  because  of  its 
inappropriateness  to  the  level  of  testing  planned,  the  in- 
convenience generated  by  the  sophisticated  matrix  originally 
conceived  as  useful,  and  the  inability  to  identify  an  existing 
benchmark  for  useful  performance  measurement.  As  a consequence, 
am  alternative  evaluative  method  was  selected:  a questionnaire 

that  would  facilitate  subject  response  to  Aid  trials  and 
that  would  still  retain  an  acceptable  measure  of  the  systematic, 
comprehensive  and  convenient  form  desired  in  an  evaluative 
tool.  We  feel  that  a multi-attribute  value  model  may  serve 
as  a useful  evaluation  tool  for  a later  test  of  the  Execution 
Aid,  as  well  as  other  Aids  to  be  identified  by  the  Project, 
in  the  more  formal  setting  of  a future  testbed  environment. 

A second  factor  consistent  with  an  informal  but  systematic 
evaluation  of  the  prototype  Aid  involved  the  operational 
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situation,  which  had  to  ba  sufficiently  realistic  in  order 
to  gain  user  acceptance  of  the  decision  analytic  methodology. 

Based  on  the  expert  advice  of  naval  personnel,  we  identified 
the  enemy  ASM  threat  as  one  that  is  highly  representative  of 
hostile  operational  situations  and  well  enough  understood  to 
serve  both  as  an  important  illustration  of  the  power  of  the 
Aid  and  as  a realistic  setting  for  the  pilot  test.  Pre- 
liminary investigation  of  the  ASM  threat  characteristics 
indicated  that  the  ASM  threat  was  both  probable  and  suffi- 
ciently predictable  to  justify  the  need  to  pre-plan  for  its 
contingent  occurrence.  Subsequent  investigation  has  rein- 
forced this  conclusion,  and  has  further  svpported  the  develop- 
ment of  a richer  threat  model  with  a high  degree  of  user 
acceptance  for  which  a "contingent  aid"  is  worthy  of  opera- 
tional development.  A full  description  of  the  threat  model 
is  contained  in  Section  2.2.2  of  this  report. 

The  final  element  of  a systematic  evaluation  of  the 
prototype  Aid  consisted  of  the  workshop  trial  itself  in 
which  a number  of  subjects  were  introduced  to  the  Aid  through 
the  pilot  test.  Given  the  early  stage  of  development  of  the 
Aid,  we  felt  the  most  promising  method  of  test,  short  of 
operational  simulation,  was  the  use  of  a workshop  trial. 

This  technique  consisted  of  exposing  a number  of  operational 
subjects  to  six  distinct  steps  in  the  test  procedure  over  a 
time  period  of  three  to  four  hours  for  each  subject.  The 
subjects  were  first  briefed  on  the  goals  of  the  ODA  Project, 
the  objectives  of  the  pilot  test  and  evaluation,  and  the 
tactical  situation  facing  the  task  force  commander  involved 
in  the  ONRODA  scenario  mission.  Next,  the  subjects  were 
introduced  to  a series  of  off-line  (vu-graph)  and  on-line 
(computer-graphic)  interactions  to  familiarize  them  with  the 
Aid  and  with  the  decision  analytic  properties  incorporated 
in  it.  These  properties  included  the  sensitivity  features 
which  enabled  full  manipulation  of  the  Aid  as  the  trial 
progressed.  During  the  final  phase  of  the  test  procedure, 
the  subject  encountered  a representative  sequence  of  threat 
events  (indicators)  in  an  on-line  interaction  with  the 
computerized  Aid.  At  the  completion  of  the  operational 
trial,  the  subject  was  queried  as  to  his  acceptance  of  the 
prototype  configuration,  that  is,  number  of  displays,  type 
of  displays,  information  content,*  and  information  logic. 

Finally,  the  subject  was  asked  to  respond  to  the  evaluative 
questionnaire.  The  results  of  the  pilot  testing  procedure 
were  useful  in  defining  ultimate  user  applications  in  both  a 
training  and  operational  sense,  as  well  as  in  producing 
numerous  suggestions  for  Aid  refinement  and  software  modification. 
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Aid  generalization  - Investigation  into  a more  generalized 
use  of  the  initial  prototype  (Execution  Aid)  has  taken  two 
directions  during  the  phase  of  research  just  completed.  In 
the  first  instance,  we  directed  our  efforts  toward  aid  en- 
hancement, specifically,  the  development  of  methods  to  model 
more  than  three  state  hypotheses  and  to  explicitly  incor- 
porate time  and  risk  as  features  of  a more  comprehensive 
aid.  In  the  second  instance,  we  concentrated  on  defining 
decision  situations  other  than  a task  force  commander's 
response  to  the  ASM  threat. 

More  than  three  hypotheses  - With  respect  to  examining 
alternatives  for  handling  tactical  situations  in  which  more 
than  three  state  hypotheses  are  of  concern,  the  following 
methods  were  investigated: 

o structuring  the  situation  in  a hierarchical  manner 
and  simplifying  each  level  of  the  hierarchy  to  a 
three- state  description; 

o displaying  the  distances  to  the  thresholds  instead 
of  a complete  representation  of  the  state  space; 

o projecting  thresholds  and  probabilities  onto 
planes  in  the  state  space;  and 

o displaying  cutting-planes  of  the  state  space. 

After  examination  of  the  relative  merits  of  each  approach, 
we  tend  to  favor  the  cutting-plane  method  for  handling 
situations  where  more  than  three  state  hypotheses  must  be 
considered.  However,  further  investigation,  including  a 
computer-graphic  implementation  is  necessary  before  a defini- 
tive recommendation  can  be  made,  particularly  as  it  pertains 
to  a graphical  representation.  A complete  description  of 
the  analysis  of  each  of  the  methods  explored  is  presented  in 
Section  3.1  of  this  report. 

Time  and  risk  - Investigations  of  time  and  risk  factors, 
as  incorporated  features  of  tactical  decision  aids,  are 
inconclusive  at  this  time.  Four  alternatives  for  addressing 
time  within  the  context  of  the  air/submarine  ASM  threat 
situation  were  examined  for  their  possible  impact  on  the 
tactical  decision  of  when  to  take  or  modify  an  action.  The 
time  alternatives  addressed  were: 

o Time  windows  for  executing  response  actions. 

o Time  projections  of  the  probability  bug. 

o Time -dependent  value  functions. 

o Date/Time  displays  for  threat  indicators  received. 
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The  time  provisions  of  date/time  tags  on  the  threat 
indicators  and  the  time-to-complete  tags  on  the  tables  of 
response  actions  have  been  incorporated  into  the  Execution 
Aid  software.  Although  these  concessions  to  th&^tlme  problem 
are  minimal#  they  are  the  only  ones  identif ied^for  software 
implementation  at  this  time.  The  methods  of  using  time- 
windows#  time  projections  of  probabilities#  and  time-dependent 
value  functions  involve  estimates  that  appear  to  be  virtually 
impossible  to  make  in  advance  with  sufficient  accuracy  to 
influence  action  selection.  Aid  implementation  in  a testbed 
environment  may  reveal  further  useful  insights  in  this  area 
of  investigation. 

An  investigation  into  the  stability  of  the  action 
thresholds#  as  currently  presented  in  the  prototype  Exe- 
cution Aid#  to  the  decision  makers'  attitude  toward  risk 
revealed  a significant  sensitivity.  As  the  value  tables  now 
structured  were  assessed  under  a condition  of  certainty  with 
respect  to  enemy  intentions#  the  thresholds  are  calculated 
on  an  expected  value  basis  which  assumes  a risk  neutral 
attitude.  Decision  analysis  literature  offers  several  alter- 
native methods  for  incorporating  risk  into  the  value  assess- 
ment. However#  it  is  not  clear  what  method#  if  any,  can  be 
usefully  incorporated  into  the  prototype  Aid  in  its  present 
form.  The  methods  examined  for  possible  adaptation  to  the 
Aid  included  the  technique  of  risk  assuming  the  form  • the 
utility  function  such  as  an  exponential;  specifying  _xsk 
attitude  in  a reference  gamble;  and  the  BRLT  procedure  based 
on  lotteries  for  different  act/state  combinations.  The 
limited  investigations  conducted  indicate  a need  to  consider 
further  the  impact  of  risk  aversion  to  the  decision  maker's 
action  selection  process.  Furthermore#  it  appears  that 
traditional  methods  of  assessing  risk  may  be  too  cumbersome 
when  applied  to  the  structure  of  a tactical  decision  aid, 
and  this  generates  the  requirement  for  new  approaches  to  the 
problem  of  modeling  risk  attitudes. 

Generalized  use  - At  the  request  of  the  Scientific 
Officer,  we  undertook  a second  direction  in  our  investi- 
gation of  aid  generalization,  specifically,  a search  for 
tactical  decision  situations  other  than  a task  force 
commander's  response  to  the  ASM  threat.  Accordingly,  the 
decision  analytic  properties  of  the  prototype  Aid  were 
examined  with  the  view  that  the  key  property  is  one  of 
probability  thresholds  most  applicable  to  situations  where 
states  are  probabilistically  independent  of  actions.  The 
ONRODA  scenario  was  re-examined,  and  the  strike  timing 
decision  was  identified  as  meeting  this  condition  with  the 
reservation  that  more  than  one  tier  of  conditioning  events 
must  be  considered.  In  addition  to  this  generalized  appli- 
cation, a variation  of  the  basic  properties  of  probability 


.r  . . . . : •“  • * • ....  . , 


viii 


and  utility  theory  (in  a combined  aid  form)  were  applied  to 
the  decision  problem  of  target  selection  confronting  the 
Commander,  Naval  Support  Forces  in  the  ONRODA  Amphibious 
Warfare  Scenario. 

Strike  timing  generalization  - A significant  result  of 
our  Aid  generalization  investigations  is  the  identification 
of  the  key  situationally-dependent  property  of  probability 
thresholds.  That  is,  the  prototype  Aid  is  designed  to 
monitor  the  tactical  environment  and  recommend  a course  of 
action  based  on  probability  thresholds.  For  this  reason, 
the  Aid  appears  best  suited  to  deal  with  operational  contin- 
gencies (events  of  possible  but  uncertain  occurrence)  that 
would  significantly  impact  mission  plan  and  execution. 

Thus,  it  is  proper  to  characterize  the  Execution  Aid  as  a 
"contingency  aid,"  and  to  seek  out  contingency  situations 
where  the  state  probabilities  are  independent  of  the  actions, 
and  the  value  of  the  actions  can  be  assessed  as  a function 
of  the  states  (that  is,  the  specification  of  an  action/state 
pair  is  a sufficient  description  of  the  situation  to  assign 
a value) . A complete  technical  description  and  application 
of  these  features  is  discussed  in  Section  3.4  of  the  Report, 
where  the  strike  mission  timing  decision  bears  on  the  over- 
all tactical  situation  and  where  several  multiple  triangular 
display  techniques  are  discussed.  The  area  in  which  decision 
analysis  appears  most  applicable  at  this  point  in  our  investi- 
gations is  that  of  tactical  response  to  contingency  situations 
involving  operational  disruption  by  uncertain  events.  The 
generalization  investigation  has  also  indicated  a wider 
applicability  of  the  prototype  Execution  Aid  to  situations 
other  than  the  ASM  threat  initially  modeled.  A preliminary 
specification  for  a strike  mission  timing  aid  has  been 
completed  for  subsequent  software  development. 

Tactical  grid  generalization  - The  investigation  described 
in  the  preceding  section  was  concerned  with  identifying  a 
more  generalized  use  of  the  properties  of  the  prototype 
Execution  Aid  which  was  described  in  detail  in  our  prior 
technical  reports.  In  contrast,  a separate  investigation 
was  conducted  in  which  a generalized  use  and  application  of 
the  underlying  principles  of  probability  and  utility  theory 
were  implemented  in  computer-graphic  form  for  aiding  tactical 
decision  making.  The  investigation  was  not  limited  to 
situations  in  which  probability  thresholds  for  triggering 
tactical  responses  were  appropriate.  Rather,  we  chose  a 
situation  where  the  task  force  commander,  faced  with  multiple 
mission  requirements,  must  select,  on  the  basis  of  expected 
value,  the  best  outcome  of  various  target  selection  decisions. 
The  tactical  grid  or  matrix  development  for  this  case 
considered  each  of  the  major  phases  of  the  amphibious  opera- 
tion presented  in  the  ONRODA  amphibious  warfare  scenario,  that 


is,  pre-assault,  assault,  and  post-assault,  in  conjunction 
with  the  major  mission  requirements  of  task  force  defense, 
support  of  the  landing  force,  and  damage- limiting  in  Grey 
nation.  Section  3.4  of  the  report  describes  in  detail  the 
model  development  and  IBM  5100  computar-graphic  implementation 
of  the  tactical  grid.  The  investigation  and  demonstration 
indicates  that  decision  analysis  principles  of  probability 
and  utility  theory  can  in  combination  be  used  on-the-spot  to 
formulate  a useful  input/output  decision  aid. 


SUMMARY  OF  ACCOMPLISHMENTS  AND  CONCLUSIONS 
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Research  Accomplishments 

o An  evaluative  questionnaire/  based  on  a list  of 

desired  decision  aid  characteristics/  was  developed 
for  use  in  the  pilot  test.  This  questionnaire 
proved  to  be  a convenient  and  efficient  instrument 
for  recording  the  subjects’  judgments  about  the 
prototype  Execution  Aid. 

o A range  of  "enemy"  ASM  threats  was  modeled  in  a 

form  that  could  be  incorporated  into  the  Execution 
Aid  for  pilot  testing.  These  threats  included  the 
small-scale  air/submarine  threat  and  the  large- 
scale  ASM  attack  system  (containing  air,  surface/ 
and  subsurface  attack  elements) . The  modeling 
form  consisted  of  a hierarchical  tiering  of  threat 
activity  and  a list  of  indicators  of  "enemy"  tac- 
tical intent,  with  the  air/submarine  model  only 
implemented  in  computer-graphic  software. 

o A pilot  test  procedure,  consisting  of  a sequence 
of  off-line  (vu-graph)  and  on-line  (interactive 
computer-graphic)  interactions  was  developed  for 
aid  evaluation.  This  procedure  proved  to  be  an 
effective  way  to  create  Aid  comprehension  and  ease 
of  use  by  the  subjects  tested. 

o Pilot  testing  was  conducted  and  the  conceptual 
acceptance  of  the  Execution  Aid  was  determined. 

In  addition,  the  pilot  testing  identified  areas 
for  software  modification  and  refinement. 

o A user's  manual  is  being  prepared  to  support 

computer-graphic  implementation  of  the  prototype 
Execution  Aid. 

] 

o An  investigation  into  methods  of  modifying  the  Aid 
to  accommodate  more  than  three  state  hypotheses 
was  completed.  Software  specifications  necessary 
to  realize  this  modification  were  identified  but 
have  not  been  implemented. 

o Initial  investigations  have  been  made  into  methods 
of  incorporating  explicit  considerations  of  time 
and  risk  into  the  Aid. 

o A method  to  display  graphically  more  than  one  tier 
of  conditioning  events  has  been  identified. 
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A resource  allocation  aid  that  utilizes  decision 
analytic  principles  has  been  developed.  Prototype 
computer  software  has  been  developed  for  this  aid 
on  the  portable  IBM  5100  microcomputer. 

o The  generalized  application  of  the  prototype 

decision  aid  to  situations  other  than  reaction  to 
the  ASM  threat  has  been  established. 

o A tentative  matching  of  prototype  decision  aids  to 
task  force  decision  situations  has  been  performed. 

Conclusions 

Our  research,  development,  and  testing  to  date  has 
resulted  in  several  conclusions  about  the  acceptability  and 
range  of  application  of  decision  analysis  to  a Navy  task 
force  commander's  mission  execution  decisions. 

The  results  of  the  pilot  testing  effort  indicate  that 
the  concepts  involved  in  our  decision  analytic  Execution  Aid 
are  acceptable  to  Navy  task  force  commanders.  These  results 
lead  us  to  conclude  that  the  prototype  Aid  is  now  ready  to 
undergo  more  thorough  evaluative  testing  in  a controlled 
experiment. 

The  investigations  of  the  ASM  attack  system  indicate 
that  this  threat  can  be  adequately  modeled  in  a Bayesian 
hierarchical  analysis,  and  the  Aid  can  usefully  monitor  the 
threat  as  it  develops. 

The  investigations  into  the  generalized  use  of  the  Aid 
indicate  that  it  is  applicable  to  a broader  class  of  decision 
situations  than  those  involving  ASM  threat  alone.  In  par- 
ticular, the  Aid  appears  to  be  applicable  to  a wide  range  of 
contingencies,  that  is,  uncertain  events  that  might  warrant 
a change  in  operational  plan.  The  probability  threshold 
concept  appears  to  be  especially  applicable  in  contingency 
situations  in  which  state  probabilities  are  independent  of 
the  possible  actions  and  value  is  a function  of  the  action/ 
state  combinations.  Additionally,  it  appears  possible  to 
extend  the  graphic  display  of  the  probability  space  to 
consider  more  than  one  tier  of  conditioning  events  and  more 
than  three  state  hypotheses  within  a tier. 

Finally,  other  classes  of  decision  aids,  characterized 
by  the  requirement  for  on-the-spot  analysis  and  expected 
value  outcome  of  alternative  decision  choices,  are  available 
for  development  and  evaluation. 


xii 


DECISION  ANALYSIS  AS  AN  ELEMENT  IN  AN 
OPERATIONAL  DECISION  AIDING  SYSTEM 
(PHASE  III) 


DD  1473 

EXECUTIVE  SUMMARY 

FIGURES 

TABLES 

1.0  INTRODUCTION 

1.1  Task  Definition 

1.1.1  Scope  of  Office  of  Naval  Research 

(ONR)  Operational  Decision  Aids 
(ODA)  project 

1.1.2  Decisions  and  Designs,  Incorporated 

(DDI)  scope  and  objective 

1.1.3  DDI  task  orientation  for  phase  III 

(FY76) 

1.2  Research  Approach 

2.0  PILOT  TESTING  FOR  AID  REFINEMENT 

2.1  Development  of  a Framework  for  Eliciting 

Test  Subjects'  Responses  to  Aid  Trials 

2.1.1  Problem  and  need  specification 

2.1.2  Alternative  approaches 

2.1.3  Status  and  conclusions 

2.2  Modeling  of  Red  ASM  Threat  Elements 

2.2.1  Problem  and  need  specification 

2.2.2  Description  of  threat  models 

2.2.3  Status  and  conclusions 

2.3  Pilot  Testing 

2.3.1  Problem  and  need  specification 

2.3.2  Description  of  pilot  test  procedure 

2.3.3  Findings  and  recommendations 

3.0  INVESTIGATION  OF  AID  GENERALIZATIONS 

3.1  More  than  Three  State  Hypotheses 


Page 

ii 

iv 

xv 

xviii 

1 


1 

1 

2 

2 

4 


5 

5 

8 


9 

10 

14 

18 

18 

18 

20 

22 

22 


xiii 


3.1.1  Status  and  conclusions 


26 

3.2  Explicit  Consideration  of  Time  26 

3.2.1  Time  windows  26 

3.2.2  Time  projections  of  the  probability 

bug  28 

3.2.3  Time-dependent  value  functions  30 

3.2.4  Time/date  display  32 

3.2.5  Status  and  conclusions  32 

3.3  Consideration  of  Risk  34 

3.3.1  Problem  and  need  specification  34 

3.3.2  Alternative  approaches  35 

3.3.3  Conclusions  and  findings  38 

3.4  Generalized  Use  38 

3.4.1  Execution  aid-contingent  mode  39 

3.4.2  Tactical  grid  for  resource 

allocation  53 

3.4.3  Status  and  conclusions  60 

4.0  SUMMARY  OF  ACCOMPLISHMENTS,  CONCLUSIONS  AND 

RECOMMENDATIONS  63 

4.1  Summary  of  Accomplishments  63 

4.2  Conclusions  64 

4.3  Recommendations  65 

4.3.1  Decision  aid  test  and  evaluation  65 

4.3.2  Research  and  development  66 

APPENDIX  A - MATCHING  AIDS  TO  SITUATIONS  68 

APPENDIX  B - TEST  PROCEDURE  ILLUSTRATIONS  75 

APPENDIX  C - QUESTIONNAIRE  87 

APPENDIX  D - INVESTIGATIONS  FOR  TREATMENT  OF  MORE 

THAN  THREE  STATE  HYPOTHESES  90 

REFERENCES  116 

DISTRIBUTION  LIST  118 


FIGURES 


Figure 

2-1: 

Single-Element  Hierarchical  Threat 
Analysis 

13 

Figure 

2-2: 

Multiple-Element  Hierarchical  Threat 
Analysis 

17  - 

Figure 

3-1: 

Three-Dimensional  State  Space  and 
Probability  Triangle 

23 

Figure 

3-2: 

Decision  Thresholds  - Value  Assess- 
ment and  Display 

25 

Figure 

3-3: 

Probability  Bug  Projections 

29 

Figure 

3-4: 

Time-Dependent  Value  Functions 

31 

Figure 

3-5: 

Probability  Distributions  of  Attack 
Time  Conditioned  on  Capability 

33 

Figure 

3-6: 

Risk  Averse  and  Risk  Neutral  Utility 
Curves 

36 

Figure 

3-7: 

Thresholds  Calculated  from  Utility 
Curves 

37 

Figure 

3-8: 

Strike  Timing  Decision — Tree  Structure 

41 

Figure 

3-9: 

Strike  Timing  Decision — Graphic 
Display  (Initial  Probability  Assess- 
ment) 

42 

Figure 

3-10: 

Strike  Timing  Decision — Revised 
Weather  Probabilities 

47 

Figure 

3-11: 

Alternative  Strike  Timing  Model — 
Tree  Structure 

50 

Figure 

3-12: 

Alternative  Strike  Timing  Model — 
Graphic  Display 

51 

Figure 

3-13: 

Targets  in  the  Amphibious  Warfare 
Scenario 

55 

Figure  B-l: 

Probability  Phase  Interpretation 

81 

Figure  B-2: 

Prior  Probability  Assessment 

82 

Figure  B-3: 

Threshold  Display 

85 

XV 


Figure  D-l: 

Hierarchy  of  Events 

Figure  D-2: 

Distance  to  Threshold  I 

Figure  D-3: 

Distance  to  Threshold  II 

Figure  D-4 : 

Probability  Tetrahedron 

Figure  D-5: 

Probability  Tetrahedron  with 
Probability  Planes 

Figure  D-6: 

Probability  Tetrahedron  with 
Probability  Planes 

Figure  D-7: 

Probability  Tetrahedron  with 
Probability  Planes 

Figure  D-8: 

Probability  Tetrahedron  with 
Probabiltiy  Planes 

Figure  D-9 : 

Probability  Tetrahedron  Showing 
Probability  Bug  Development 

Figure  D-10: 

Probability  Tetrahedron  Showing 
Probability  Bug  Development 

Figure  D-ll: 

Probability  Tetrahedron  Showing 
Probability  Bug  Development 

Figure  D-12; 

Probability  Tetrahedron  Showing 
Probability  Bug  Development 

Figure  D-13: 

Location  of  Probability  Bug, 
State  1 

Figure  D-14: 

Location  of  Probability  Bug, 
State  2 

Figure  D-15: 

Location  of  Probability  Bug, 
State  3 

Figure  D-l 6 s 

Location  of  Probability  Bug, 
State  4 

Figure  D-17: 

Relative  Locations  of  Probability 
Bug,  States  1 through  4 

Figure  D-18: 

Projection  Method 

Figure  D-19: 

Cutting-Plane  Method 

Figure  D-20: 

Cutting-Plane  Method 

Page 

Figure  D-21s 

Initial  Threshold 
the  2-3-4  Cutting 

Calculation 

Plane 

for 

112 

Figure  D-22: 

Revised  Threshold 
the  2-3-4  Cutting 

Calculation 

Plane 

for 

113 

Figure  D-23: 

Revised  Threshold 
the  Cutting-Plane 

Calculation 

Method 

for 

114 

XV  ii 


TABLES 


Table  3-5:  Alternative  Strike  Timing  Model — 

Value  Tables 

Table  3-6:  Expected  Net  Target  Value  Calculation — 

First  Sortie 

Table  3-7:  Target  Selection  Problem — Inputs 

Table  3-8:  Target  Selection  Problem — Pre-Assault 

Phase 

Table  3-9:  Target  Selection  Problem — Assault  and 

Post-Assault  Phases 

Table  A-l:  Matching  Analytic  Options  to  Situations 

Table  A-2:  Matching  Decision  Aids  to  Situations 

Table  B-l:  Possible  Red  Operational  Options 

Table  B-2:  Red  ASM  Threat  Indicators 

Table  B-3:  Possible  Blue  Response  Options 

Table  B-4:  Commander's  Concerns 


Decision  Threshold  Calculation 


Multi-Attribute  Value  Matrix 


Table  B-6: 


Four-State  Value  Table 


Table  D-l 


1.0  INTRODUCTION 


Task  Definition 


1.1.1  Scope  of  Office  of  Naval  Research  (ONR)  Opera- 
tional Decision  Aids  (PDA)  project  - *fhe  objective  otihs  N 
ONR  project  for  the  research  and  development  of  operational^ 
decision  aids  has  remained  virtually  unchanged  since  the 
program's  inception.  The  project  remains  oriented  toward 
aiding  naval  task  force  commanders  in  the  planning  and 
execution  of  naval  missions  using  relevant  man-machine 
technologies  in  an  interactive  computer-graphic  mode  as  an 
integral  part  of  an  afloat  tactical  command  center.  The 

most  immediate  naval  command  system  program  for  implementation 
of  a proven  decision  aiding  sub-system  remains  the  sea-based 
tactical  flag  command  center  (TFCC) . A prototype  of  this 
command  center  is  currently  in  the  concept  definition  phase 
of  development  by  NAVELEX. 

The  next  major  ODA  project  undertaking  will  be 
the  completion  of  a project  testbed  for  prototype  decision 
aid  test  and  evaluation  under  simulated  conditions  of  battle 
management.  The  lead  unclassified  testbed  facility  for  the 
planned  September  1976  decision  aid  trials  is  the  Wharton 
Decision  Sciences  Computer  Center  of  the  University  of 
Pennsylvania.  Several  classified  command  and  control  test- 
bed facilities  now  in  the  planning  stages  are  currently 
under  consideration  for  subsequent  decision  aid  testing 
where  the  use  of  classified  data  will  be  required  in  support 
of  tactical  simulation  for  the  purpose  of  aid  evaluation. 

1.1.2  Decisions  and  Designs,  Incorporated  (DPI)  scope 
and  objective  - As  reported  in  DPI  Technical  Reports  74-2 
and  75-13*,  DDI  has  focused  research  and  investigation  on 
the  decision  environment  of  naval  task  force  commanders  in 
order  to  identify,  to  the  extent  possible,  classes  of  decision 
problems  for  which  decision  analytic  properties  can  be 
applied  as  decision  aiding  tools.  To  date,  a compact  set  of 
aid  components  to  support  operational  planning  and  execution 
has  been  identified  for  preliminary  specification  and  computer 
implementation;  the  current  task  of  prototype  aid  develop- 
ment is  directed  toward  the  contingency  planning  and  mission 
execution  phase  of  naval  task  force  operations.  Prototype 
decision  aids  of  this  category  will  be  evaluated  later  this 
year  in  the  Wharton  School  testbed. 


Brown,  Hoblitzell,  Peterson,  and  Ulvila  (1974)  and  Brown 
et.  al.  (1975). 
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Although  a more  generalized  use  of  the  types  of 
decision  aids  currently  identified  is  desired  as  part  of  the 
research  effort,  our  recent  direction  has  been  to  refine  a 
single  computer-graphic  prototype  for  test  and  evaluation  by 
the  Navy  as  noted  above.  The  investigation  into  adapting 
prototype  aid  characteristics  to  an  enlarged  number  of 
decision  classes/problems  will  continue  as  outlined  in  our 
current  technical  proposal. 

1.1.3  DPI  task  orientation  for  Phase  III  (FY76)  - As 
specified  in  DDI  technical  Proposal  4j-7b  of  3 October  1975, 
the  DPI  task  undertaking  for  the  Phase  III  research  effort 
centered  on  five  areas  of  investigation: 

o Pevelopment  of  decision  aid  evaluation  criteria; 

o Aid  testing  for  conceptual  soundness; 

o Modification  and  refinement  of  Aid  software  and 
preparation  of  a user's  manual; 

o Anti-ship  missile  (ASM)  threat  modeling; 

o Aid  generalization  (enhancement)  investigations. 

Subsequent  tasking  by  the  Scientific  Officer 
added  the  following  areas  of  investigation: 

o Generalized  use  of  the  Aid  in  tactical  situations 
other  than  ASM  threat; 

o Preliminary  matching  of  prototype  aids  to  tactical 
decision  situations. 

Sections  2.0  and  3.0  of  this  report,  as  well  as 
the  Appendices,  provide  substantive  reports  of  our  research 
on  each  of  the  areas  cited  above.  As  in  previous  investi- 
gations, the  major  focus  of  the  PPI  effort  has  been  in  the 
area  of  technology  development,  that  is,  a continuing  search 
for  and  refinement  of  decision  analytic  tools  applicable  to 
the  tactical  decision  environment  under  investigation  for 
the  purpose  of  identifying  useful  aids  of  an  interactive 
computer-graphic  form.  There  is  an  apparent  need  for  evalua- 
tion of  various  prototype  aids  through  trials  that  will 
stimulate  user-developer  interaction.  It  is  our  view  that 
only  the  actual  use  of  candidate  aids  will  verify  their  full 
potential  as  decision  aiding  tools  and  encourage  a satis- 
factory level  of  operational  acceptance. 

1. 2 Research  Approach 

The  pilot  testing  activity  was  conducted  in  the  form  of 
structured  workshop  trials  in  which  the  experimenter,  a DDI 
analyst,  and  a Naval  subject  with  operational  experience 
exercised  the  prototype  execution  aid  in  a simulated  threat 
situation  utilizing  an  interactive  computer-graphics  terminal 
After  each  pilot  testing  session,  the  subject  provided  his 
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judgments  on  the  strengths  and  weaknesses  of  the  Aid  by 
means  of  a questionnaire.  Since  the  purpose  of  the  pilot 
test  was  to  establish  the  conceptual  acceptability  of  the 
Aid  and  its  strengths  and  weaknesses  as  a basis  for  refining 
the  computer  software  rather  than  to  formally  evaluate  the 
Aid,  the  test  was  not  conducted  as  a controlled  psychological 
experiment. 

The  ASM  threat  element  modeling  was  accomplished  through 
interviews  with  knowledgeable  Naval  personnel  and  through  a 
search  of  the  relevant  literature. 

Mathematical  analysis  was  used  to  identify  possible  Aid 
generalizations  for  the  modeling  of  more  them  three  state 
hypotheses,  time,  and  risk.  Workshop  trials  were  used  to 
evaluate  the  alternatives  identified  in  each  area  as  well  as 
to  explore  the  more  generalized  use  of  the  Aid.  These 
workshop  trials  and  the  applied  experience  of  the  authors 
were  used  to  perform  the  matching  task. 

Finally,  a major  part  of  the  effort  in  the  contractual 
phase  involved  software  programming  of  the  Aid  refinements 
that  were  identified. 


2.0  PILOT  TESTING  FOR  AID  REFINEMENT! 


Our  research  in  Phase  II  of  the  ODA  project  produced  an 
initial  prototype  of  a computerized,  interactive,  graphic 
decision. aid  for  the  support  of  a naval  task  force  commander 
engaged  in  mission  execution.2  As  with  most  prototypes, 
this  initial  computer-graphic  Aid  required  preliminary 
evaluation  in  order  to  further  develop  and  refine  the  soft- 
ware specifications.  This  section  of  the  report  describes 
the  evaluation  procedure  undertaken  to  identify  modifications 
and  refinements  that  would  enhance  user  acceptance  and  to 
strengthen  the  Aid  in  preparation  for  a more  rigorous  test 
and  evaluation  in  the  forthcoming  project  testbed  environment. 

The  preliminary  Aid  evaluation  involved  the  following 
procedural  steps: 

1.  Identification  of  a list  of  desired  Aid  charac- 
teristics. 

2.  Preparation  of  a realistic  testing  scenario 
(operational  situation) . 

3.  Performance  of  a pilot  test. 

4.  Evaluation  of  test  results  for  incorporation 
of  the  refinements  into  prototype  computer 
software. 

The  following  sections  of  this  report  describe  the  evaluation 
process  according  to  the  above  steps:  Section  2.1  describes 

the  development  of  a list  of  desired  Aid  characteristics  in 
the  form  of  a questionnaire  that  is  based  upon  multi-attribute 
evaluation  criteria;  Section  2.2  describes  the  development 
of  a testing  scenario  based  upon  a model  of  the  Soviet  anti- 
ship missile  (ASM)  threat;  and  Section  2.3  describes  the 
pilot  testing  activity  and  the  findings  obtained  from  the 
test. 


!Throughout  this  report,  the  authors  assume  that  the  reader 
is  familiar  with  the  description  of  -he  Execution  Aid  that 
is  contained  in  Section  3 of  Brown  et  al.  (1975).  While  an 
attempt  is  made  in  this  report  to  provTcTe  the  reader  with 
a recapitulation  of  the  critical  features  of  the  Aid,  we 
suggest  that  the  serious  technical  reader  review  the  material 
as  originally  presented. 

2This  aid  is  described  in  Brown  et  al.  (1975) . Hereafter, 
this  aid  will  be  referred  to  as  the  Execution  Aid  or  Aid. 
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2.1  Development  of  a Framework  for  Eliciting  Test  Subjects' 

Responses  to  Aid  Trials 

In  principle,  multi-attribute  evaluation  criteria  can 
be  formed  into  a template  which  can  then  be  used  to  evaluate 
a decision  aid  such  as  the  Execution  Aid.  Such  a template 
would  involve  identifying  performance  measures,  weighting 
them  according  to  their  relative  importance  in  Navy  mission 
execution  (operational)  situations,  and  evaluating  the 
decision  aid  along  each  criterion  dimension.  In  pursuing 
the  development  of  this  scoring  or  evaluation  technique,  we 
concluded  that  the  effort  required  to  develop  an  accurate 
template  was  not  warranted  for  the  purposes  of  this  preliminary 
evaluation.  Accordingly,  the  following  sections  describe  an 
alternative  method  for  aid  evaluation  using  a set  of  evalua- 
tive criteria  transformed  into  a questionnaire  appropriate 
for  the  pilot  testing  objective.  Guidance  is  also  provided 
to  indicate  the  actions  necessary  to  convert  the  question- 
naire into  a multi-attribute  evaluation  tool  in  support  of  a 
more  formal  test  and  evaluation. 

2.1.1  Problem  and  need  specification  - The  pilot  test 
demanded  a systematic,  comprehensive  and  convenient  vehicle 
for  identifying  areas  of  strength  and  weakness  in  the  Exe- 
cution Aid.  Earlier  attempts  to  identify  any  conceptual 
shortcomings  in  the  Aid  were  carried  out  in  a very  informal 
manner  resulting  in  a more  or  less  random  list  of  comments. 
Thus,  a systematic,  evaluative  framework  was  required  to 
document  changes  for  Aid  improvement.  However,  important 
performance  areas  were  not  to  be  overlooked.  The  framework 
needed  to  be  comprehensive  as  well  as  systematic.  Finally, 
the  framework  needed  to  be  convenient  in  order  to  be  useful 
for  our  particular  pilot  testing  situation;  that  is,  the 
nature  of  the  pilot  test  required  that  the  test  subjects  be 
responsive  during  the  elicitation  phase  of  the  test  without 
a great  deal  of  prior  preparation. 

2.1.2  Alternative  approaches  - We  investigated  two 
alternative  approaches  in  our  efforts  to  identify  a system- 
atic, comprehensive,  and  convenient  framework  for  eliciting 
test  subjects'  responses.  First,  we  investigated  the  use  of 
a template  of  multi-attribute  evaluation  criteria.  Next,  we 
investigated  a questionnaire  that  was  essentially  a deriva- 
tive of  the  more  complex  multi-attribute  model. 

Our  investigation  of  the  multi-attribute  value 
model  began  by  considering  the  notional  illustration  pre- 
sented in  Brown,  et  al.  (1975),  which  is  shown  in  Table  2-1. 

In  this  illustration,  three  aids,  "Min,"  "Mod,"  and  "Max, "3 
are  evaluated  for  the  "choice  of  strike  plan"  decision 


3These  aids  are  described  in  Section  2 of  Brown,  et  al.  (1975) . 


SITUATION  SPECIFIC  EVALUATION  OF  GIVEN  AID  CHOICE 

(NOTIONAL  ILLUSTRATION) 

SITUATION  * STRIKE  CHOICE-ONROOA  SCENARIO 


AID  COMPLEXITY 


MIN 


MOD 

DECOMPOSED 


MAX 


CRITERIA 

PREFORMAL 

RANK 

ORDERING 

VALUATION 

CONDITIONED 

ASSESSMENT 

EXTENSIVE 

STEP-THRU 

CRITERION 

WEIGHT 

LOGIC  OF  CHOICE 

CONCEPTUAL  COMPLETENESS 

1 

2 

3 

10 

DISAGGREGATION 

1 

2 

3 

10 

SOUND  PREDICTIONS 

- 

- 

- 

- 

LOGICAL  CONCLUSIONS 

— 

— 

QUALITY  OF  INPUT 

DATA  GATHERING 

2 

3 

2 

MANAGEMENT  OF  STAFF  EXPERTISE 

1 

3 

2 

10 

POSING  MEANINGFUL  QUESTIONS 

3 

3 

1 

10 

SPEED  OF  RESPONSE 

ABILITY  TO  PRE-DIGEST 

— 

— 

- 

EASE  OF  ASSESSMENT 

3 

2 

1 

CALCULATION  SPEEO 

3 

2 

1 

1 

TRANSPARENCY  OF  OUTPUT 

- 

— 

— 

COSTS 

COMPUTATIONAL  NEEOS 

3 

2 

1 

1 

PSYCHOLOGICAL  DISCOMFORT 

3 

3 

1 

2 

SIMPLIFIED  COMMUNICATION 

3 

3 

1 

2 

SITUATION  DEPENDENCE  OF  AIO 

3 

3 

1 

1 

REQUIREO  EXPERTISE/USER  TRAINING 

3 

3 

1 

7 

PRE-CHOICE  PROCESSES 

MONITORING  THE  ENVIRONMENT 

- 

- 

- 

IDENTIFYING  DECISIONS 

- 

- 

- 

OPTION  GENERATION 

- 

- 

- 

DECISIVENESS  IN  ADORESSING  CHOICE 

- 

- 

- 

POST-CHOICE  PROCESSES 

COMMUNICATION  OF  DECISION 

1 

3 

1 

4 

JUSTIFICATION  OF  DECISION 

1 

3 

1 

4 

EVALUATION  OF  OECISION  QUALITY 

1 

3 

1 

3 

ACCEPTABILITY  OF  CONCLUSIONS 

1 

3 

2 

4 

ORGANIZATIONAL  IMPACT 

IMPROVING  INFORMATION  PROCESSING 

1 

3 

2 

SENSITIVITY  ANALYSIS 

1 

3 

1 

IDENTIFYING  IMPORTANT  INFORMATION 

1 

3 

2 

IMPROVING  DESIGN  OF  C2  SYSTEM 

1 

3 

1 

1 

KEEPING  CTF  AT  PROPER  LEVEL 

1 

3 

1 

1 

IMPROVING  TACTICAL  DOCTRINE 

1 

2 

1 

MANAGING  SUBORDINATES 

- 

- 

- 

ENSURING  COMMAND  CONTINUITY 

1 

3 

2 

OVERALL 

VALUE 


161 


252 


117 


Table  2-1: 


MULTI-ATTRIBUTE  VALUE  MODEL 
NOTIONAL  ILLUSTRATION 


■a , - 


within  the  ONRODA  scenario.4  The  first  step  in  the  evalu- 
ation is  to  indicate  the  relative  importance  of  the  criteria 
for  the  situation  by  assigning  criterion  weights.  Next, 
each  aid  is  scored  with  respect  to  each  criterion.  Finally, 
an  overall  value  is  calculated  by  taking  a weighted  sum  of 
scores  for  each  aid.  The  aid  that  has  the  highest  overall 
value  is  the  preferred  aid  for  the  situation.  In  this 
particular  illustration,  the  "Mod"  aid  is  preferred. 

Upon  further  investigation,  we  concluded  that 
this  sophisticated  method  provided  a systematic  framework 
for  eliciting  subjects'  responses,  and  that  a list  of  some 
32  criteria  was  indeed  comprehensive.  However,  the  multi- 
attribute value  model  did  not  appear  to  be  convenient  for 
use  in  the  pilot  test  undertaken.  In  particular,  the  model 
was  inconvenient  on  three  counts:  It  included  too  many 

criteria  (which  were  often  phrased  in  terms  that  would  be 
unfamiliar  to  the  subjects) ; it  required  a benchmark  that 
did  not  exist;  and  it  was  designed  to  compare  aids  rather 
than  to  examine  the  potential  value  of  a single  aid. 

Based  upon  our  test  plan  procedure  and  the 
overall  availability  of  our  test  subjects,  we  estimated  that 
one  hour  would  be  optimum  for  eliciting  evaluative  responses 
from  the  test  subjects.  Thus,  there  would  not  be  sufficient 
time  and  prior  preparation  to  elicit  responses  on  all  32 
value  criteria.  This  problem  was  remedied  by  reducing  the 
list  to  four  primary  criteria  and  phrasing  them  in  a manner 
that  would  be  more  familiar  to  the  subjects.  The  discrete 
criteria  chosen  were: 

1.  Timeliness  of  decision; 

2.  Quality  of  decision; 

3.  Ease  of  operation  of  the  Aid; 

4.  Ease  of  understanding  of  the  Aid. 

There  is  no  readily  apparent  decision  aiding 
system  in  current  use  in  the  Navy  that  could  serve  as  a 
benchmark  against  which  to  measure  the  Execution  Aid. 

Thus,  our  investigation  sought  to  uncover  an  alternative 
that  would  be  both  familiar  to  all  of  our  subjects  and 
sufficient  to  determine  a scale  of  measurement.  After 
searching  for  several  alternatives,  including  ships  with  and 
without  NTDS , various  specific  ships,  and  historical  versus 
new  ships,  we  concluded  that  a suitable  scale  could  not  be 
constructed  in  a useful  way  without  introducing  artifici- 
alities of  little  value. 


4The  ONRODA  scenario,  the  base  scenario  for  this  project, 
is  described  in  Payne  and  Rowney  (1975) . 


Unless  a suitable  absolute  scale  can  be  con- 
structed/ the  multi-attribute  value  model  can  only  provide 
information  on  the  relative  strengths  and  weaknesses  of 
alternatives.  This  presented  an  additional  problem  since 
we  were  attempting  to  evaluate  a single  aid.  As  an  alter- 
native/ we  considered  developing  and  comparing  three  ver- 
sions of  the  Execution  Aid  that  had  different  degrees  of 
capability: 


A minimum  version  of  the  Aid  that  would  simply 
present  threat  (situation)  data; 


o A moderate  version  that  would  present  threat 

data  plus  a probabilistic  interpretation  of  the 
data; 


o A full  version  that  would  contain  all  of  the 
Aid's  features,  that  is,  threat  data,  probabi- 
listic interpretation  of  the  data  together  with 
a probability  threshold  interpretation  of  the 
subject's  values. 

We  felt,  however,  that  such  a comparison  would  not  meet  the 
objectives  of  the  test.  On  the  other  hand,  since  the  develop 
ment  of  the  three  versions  of  the  Aid  would  help  to  explain 
the  Aid's  features  to  the  subjects,  these  versions  were 
developed. 

Reviewing  the  problems  aforementioned  in  light 
of  the  purpose  of  the  experiment — to  identify  software 
refinements — we  concluded  that  a multi-attribute  value  model 
would  hinder  rather  than  help  the  evaluative  effort.  A 
questionnaire  could  better  serve  the  objective  of  the  pilot 
test:  It  would  facilitate  elicitation,  yet  remain  as  system- 

atic and  comprehensive  as  possible.  Accordingly,  the  ques- 
tionnaire was  designed  around  the  four  primary  criteria 
identified  above,  and  its  questions  were  phrased  in  terms 
familiar  to  the  subjects. 

2.1.3  Status  and  conclusions  - The  questionnaire  was 
developed  and  used  in  the  pilot  test,  and  the  multi-attribute 
value  model  was  not  developed  beyond  the  notional  stage. 

The  questionnaire  is  presented  as  Appendix  C of  this  report. 

While  our  investigation  concluded  that  the 
questionnaire  was  the  appropriate  framework  for  eliciting 
test  subjects'  responses,  we  feel  that  a multi-attribute 
value  model  might  be  appropriate  for  a later  test  of  the 
Execution  Aid  in  a more  formal  setting.  Recall  that  the 
most  serious  problem  with  the  multi-attribute  value  model 
was  that  we  were  unable  to  identify  and  construct  a meaning- 
ful evaluation  scale.  If  subsequent  tests  are  designed  to 
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measure  the  Execution  Aid's  performance  compared  with  that 
of  another  aid,  then  the  evaluation  can  be  performed  on  a 
relative  scale  and  the  scaling  problem  is  overcome.  Our 
current  investigation  considered  only  one  aid,  so  a relative 
scale  would  have  been  meaningless;  in  a future  test,  for 
instance  at  the  project's  testbed,  several  aids  might  be 
meaningfully  compared  on  a relative  value  scale. 

2. 2 Modeling  of  Red  ASM  Threat  Elements 

In  order  to  assess  the  operational  acceptance  of  the 
Execution  Aid  by  a naval  user,  the  Aid  was  demonstrated  in 
a realistic  setting.  In  Brown  et  al.  (1975) , we  identified 
the  anti-ship  missile  (ASM)  threat  as  one  that  is  serious, 
probable,  and  predictable  enough  to  support  the  development 
of  the  prototype  Execution  Aid.  At  that  time,  we  also 
sketched  the  form  that  a "model"  of  this  particular  threat 
could  take.  However,  it  was  recognized  that  a more  detailed 
modeling  effort  was  needed  before  it  would  be  in  a form 
acceptable  to  a naval  commander.  The  following  sections 
describe  our  investigations  of  the  ASM  threat:  First,  the 

air/submarine  combination  and,  second,  a full-scale  combi- 
nation comprised  of  air,  surface,  and  sub-surface  attack 
elements. 


2.2.1  Problem  and  need  specification  - In  order  to 
properly  evaluate  the  conceptual  acceptability  of  the  Execu- 
tion Aid,  and  in  order  to  extract  meaningful  suggestions  for 
refinements  from  naval  subjects,  the  Aid  was  used  to  address 
a realistic  Navy  problem  presented  to  the  subjects  in  a 
realistic  manner.  Based  on  the  advice  of  expert  naval  per- 
sonnel, we  identified  the  ASM  threat  as  one  that  is  highly 
representative  of  hostile  operational  situations  and  well 
enough  understood  to  serve  as  both  an  important  illustration 
of  the  Aid's  power  and  a useful  setting  for  our  pilot  test. 

In  addition,  our  previous  investigation  into  the  ASM  threat 
characteristics  yielded  a tentative  conclusion  that  the 
ASM  threat  was  both  probable  and  sufficiently  predictable  to 
justify  the  effort  needed  to  pre-plan  for  its  contingent 
occurrence  in  the  course  of  naval  mission  prosecution. 

Our  initial  investigation  resulted  in  a coarse 
model  of  a single  element  of  the  ASM  threat,  the  air/sub- 
marine combination.  This  model,  however,  was  not  sufficiently 
rich  to  serve  as  a realistic  illustration  of  the  Aid.  Thus, 
further  investigation  was  indicated  in  two  areas.  First,  a 
more  detailed  model  of  the  air/submarine  threat  was  needed 
to  serve  as  a setting  for  the  pilot  test.  In  addition,  a 
model  of  the  realistic,  more  probable  threat  environment  was 
needed  to  serve  as  the  basis  for  future  tests,  to  demon- 
strate a wider  application  of  the  methodology  and  thus  to 
improve  its  conceptual  acceptability. 
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2.2.2  Description  of  threat  models  - Working  with  Navy 
intelligence  and  tactical  warfare  experts,  we  developed 
detailed  models  of  two  versions  of  the  "enemy”  ASM  threat. 
The  first  is  a model  of  the  air/submarine  strike  element  in 
which  the  air  unit  serves  as  a targeting  platform  for  the 
submarine,  the  attack  platform.  The  second  is  a model  of  a 
comprehensive  large-scale  attack  system  which  includes  air, 
surface,  and  sub- surface  attack  elements  and  is  a threat  of 
more  serious  concern  to  Navy  task  force  commanders. 

Recall  the  air/submarine  threat  described  in 
Chapter  3 of  Brown,  et  al.  (1975).  Red's  hostile  intent 
against  Blue  can  be  cKaracterized  by  three  states: 

1.  Intent  to  engage  in  routine  surveillance  only; 


2.  Intent  to  fully  prepare  to  attack  the  Blue  Task 
Force  (stopping  short  of  an  actual  attack) ; 

3.  Intent  to  attack  the  Blue  Task  Force. 

The  probabilities  of  these  intents  could  be  inferred  from  a 
set  of  indicators  using  a Bayesian  probability  updating 
technique.  Table  2-2  shows  the  indicators  developed  in  the 
original  model  and  how  they  were  used  to  update  the  proba- 
bilities of  intents.  An  assessment  is  made  of  the  prior 
probabilities  of  the  intents  and  of  the  set  of  likelihood 
ratios  for  the  indicators.  The  indicators  are  then  used  to 
update  the  probabilities  using  Bayes'  theorem.  As  shown  in 
Table  2-2,  the  probabilities  of  routine  surveillance,  full- 
scale  preparation,  and  hostile  action  are  .06,  .77,  and  .17, 
based  upon  prior  probabilities  of  .90,  .09,  and  .01  and  the 
first  four  indicators.5 

The  model  illustrated  above  was  the  starting 
point  for  our  investigation  of  threat,  and  it  indicates  the 
form  that  was  desired  for  the  output  of  the  more  detailed 
model.  That  is,  the  output  of  the  threat  model  should  be  in 
the  form  of  a list  of  indicators  and  likelihood  ratios  that 
could  be  used  to  update  prior  probabilities.  For  the  indica- 
tors shown  in  Table  2-2,  the  likelihood  ratios  were  assessed 
directly.  However,  further  investigation  of  the  threat  re- 
vealed that  the  direct  assessment  technique  was  not  work- 
able: It  was  very  difficult  to  relate  the  data  directly  to 

the  hypotheses.  Thus,  we  decided  to  use  the  analytic  tech- 
nique of  Bayesian  hierarchical  analysis. 


5This  Bayesian  probability  updating  technique  is  explained 
in  detail  in  Chapter  3 of  Brown,  fi£  al.  (1975). 
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Table  2-2:  ORIGINAL  THREAT  MODEL  INDICATOR  LIST 


In  Bayesian  hierarchical  analysis,  the  complex 
problem  of  relating  data  to  hypotheses  is  simplified  by 
introducing  an  intermediate  variable  and  assessing  the  prob- 
abilities that  link  the  data  to  the  intermediate  variable 
and  those  that  link  the  intermediate  variable  to  the  hypoth- 
eses. If  the  intermediate  variable  is  chosen  properly,  this 
procedure  is  much  easier  than  attempting  to  assess  the 
probabilities  that  link  the  data  directly  to  the  hypotheses.6 

The  structure  of  the  Bayesian  hierarchical  model 
that  was  developed  for  the  Red  air/submarine  threat  is  pre- 
sented in  Figure  2-1.  In  this  model,  a series  of  off-scene 
and  on-scene  activities  and  their  associated  observations 
that  might  be  available  to  a task  force  commander  were  pos- 
tulated. Then,  probabilities  were  assessed  linking  the 
activities  to  the  hypotheses  of  routine,  feint,  and  attack, 
and  probabilities  were  assessed  linking  the  observations  to 
the  activities.  For  example,  consider  the  targeting  activity. 
We  hypothesized  that  Red's  targeting  activity  might  occur  in 
one  of  three  ways,  intermittently,  steadily,  or  not  at  all. 

If  Red  intended  no  attack,  it  was  most  likely  that  he  would 
not  engage  in  any  targeting,  and  we  assessed  probabilities 
of  .10,  .05,  and  .85  for  the  activities  of  intermittent, 
steady,  and  none  (as  shown  in  the  top  matrix  in  Figure  2-1). 

If  Red  intended  feint,  it  was  most  likely  that  he  would 
target  in  a steady  fashion;  thus,  we  assessed  probabilities 
of  .27,  .68,  and  .05  for  intermittent,  steady,  and  none. 
Finally,  if  Red  intended  to  attack,  it  was  most  likely  that 
he  would  try  to  conceal  his  target  signal  by  using  it  inter- 
mittently, and  we  assessed  probabilities  of  .71,  .28,  and 
.01  for  intermittent,  steady,  and  none. 

In  a similar  manner,  the  bottom  matrix  of  Figure 
2-1,  linking  observations  to  activities,  was  assessed.  If 
Red  actually  was  targeting  intermittently  (an  activity) , it 
was  most  likely  that  the  task  force  commander  would  receive 
a report  that  Red  was  not  targeting  (an  observation).  Thus, 
we  assessed  probabilities  of  .30,  0,  and  .70  for  the  observa- 
tion of  intermittent,  steady,  and  none.  Further,  we  assessed 
probabilities  of  .10,  .60,  and  .30  for  the  observation  of 
intermittent,  steady,  and  none  given  that  Red's  activity  was 
steady  targeting  and  probabilities  of  .04,  .01,  and  .95  for 
observations  of  intermittent,  steady,  and  none  if  Red  was 
not  targeting. 


6Chapter  14  of  Han<tt>ook  for  Decision  iUialysis  (1973)  contains 
a technical  description  of  Bayesian  hierarchical  analysis. 
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Figure  2-1 

SINGLE  ELEMENT  HIERARCHICAL  THREAT  ANALYSIS 


Multiplying  the  lower  matrix  of  Figure  2-1, 
which  links  the  observations  to  the  activitiee,  by  the  upper 
matrix. 7 which  links  the  activities  to  the  hypotheses, 
yields  a product  matrix  that  relates  the  observations  to  the 
hypotheses.  Table  2-3  shows  this  multiplication  for  the 
targeting  example  explained  above.  This  product  matrix  pro- 
vides output  in  the  desired  form  of  an  indicator  list  re- 
lating observations  to  hypotheses.  Table  2-4  shows  the 
product  matrices  for  all  of  the  observations  in  the  hierar- 
chical model  of  the  air/submarine  version  of  the  threat. 

The  model  of  the  large-scale  attack  system  is 
also  a Bayesian  hierarchical  analysis.  Figure  2-2  shows  the 
form  of  this  model.  For  classification  reasons,  the  details 
of  this  model  cannot  be  described  in  this  report,  and  we  are 
currently  exploring  the  possibility  of  preparing  a separate, 
classified  document  describing  the  model  for  the  purpose  of 
supporting  the  later  test  and  evaluation  of  the  Execution 
Aid  in  a classified  testbed  environment.  However,  the 
methodology  for  generating  a model  of  the  large-scale  ASM 
threat  is  in  itself  a contributory  step  and  can  be  applied 
in  software  form  as  a supporting  decision  aid  module. 

2.2.3  Status  and  conclusions  - The  air/submarine 
version  of  the  threat  has  been  programmed  as  part  of  the 
Execution  Aid  software,  and  it  was  used  in  the  pilot  test. 
The  large-scale  version  of  the  threat  has  been  modeled  on 
paper  but,  because  of  the  sensitive  activity  portrayed,  the 
model  is  classified.  A computerized  implementation  of  this 
Bayesian  threat  model  is  proposed  in  the  form  of  a hierar- 
chical inference  model  in  our  current  proposal.  This  clas- 
sified model  will  be  compatible  with  our  evolving  classified 
computer  center  facility.  We  are  currently  investigating 
the  possibility  of  preparing  an  unclassified  version  that 
will  maintain  the  essence  of  the  threat  model. 


We  feel  that  the  Bayesian  hierarchical  model 
that  has  been  developed  for  the  large-scale  threat  is  a 
useful  model  that  describes  the  ASM  threat  accurately. 
Furthermore,  we  feel  that  the  model  sufficiently  represents 
the  "enemy”  ASM  threat  as  to  make  further  modeling  unnec- 
essary at  this  time  in  order  to  demonstrate  the  usefulness 
and  operational  suitability  of  the  Execution  Aid.  While 
there  is  evidence  that  the  threat  may  vary  somewhat  with 
location  and  tactics,  we  feel  that  most  of  the  variations 
are  merely  subsets  of  the  comprehensive  threat  model  and  can 
be  handled  in  the  generic  case. 


7See  any  linear  algebra  book,  for  instance  Noble  (1969), 
for  an  explanation  of  matrix  multiplication. 


Intent 

Surveil- 

Indicators ' * 

lance 

Faint 

Attack 

Political  - 

Boatlie 

- 

Normal 

- 

Ignoring 

National 

General  Alert 

- 

Theatre  Alert 

- 

Selective  Alert 

- 

No  Alert 

Deployment  - 

Tactical 

- 

Strategic 

Surface 

Normal 

- 

Attack 

• 

Withdraw 

Coons 

Incraaae/Hlgh 

- 

Normal/High 

- 

Incraaae/Low 

- 

Normal/Low 

Submarine  - 

Normal 

- 

Multiple 

Search 

Yea 

- 

No 

Targeting  - 

MW/Int. 

- 

MW/Steady 

None 

Track 

Routine 

- 

Attack 

Beacon  Teat 

- Yea 

- No 

Guidance  Radar  - Tea 
- No 

Data  Link 

- Yea 

- No 

Git  and  Coau  - Yea 
- No 

Sounda 

- Yea 

- No 

Intercept 

- Yea 

- No 

Table  2-4:  PRODUCT  MATRICES  (AIR/SUBMARINE  THREAT) 
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2. 3 Pilot  Testing 


The  pilot  testing  was  designed  exclusively  to  identify 
areas  for  software  modification  and  refinement  and  to  estab- 
lish the  degree  of  conceptual  acceptance  of  the  Execution 
Aid  by  a representative  group  of  naval  users.  Consequently, 
the  testing  activity  focused  on  creating  an  evaluative  en- 
vironment in  which  the  subjects  could  provide  informed 
judgments  on  the  strengths  and  weaknesses  of  the  Aid  rather 
than  an  experimental  environment  in  which  a subject's  per- 
formance in  the  use  of  the  Aid  was  measured  under  controlled 
conditions.  The  following  sections  describe  the  concept  of 
the  pilot  test,  its  procedures,  and  findings. 


2.3.1  Problem  and  need  specification  - As  the  prototype 
Execution  Aid  was  conceived  and  partially developed  during 
our  previous  contractual  phase,  we  were  uncertain  about  the 
degree  of  the  Aid's  acceptability  to  naval  users.  In  addi- 
tion, we  recognized  that,  as  with  all  new  technology,  the 
Aid  would  most  probably  require  modification  before  it  was 
ready  to  be  rigorously  evaluated  in  a testbed  environment. 
Given  the  Aid's  early  stage  of  development,  we  felt  that  the 
most  promising  method  of  test  would  be  a structured  workshop 
trial  in  which  a subject  with  naval  experience  would  exercise 
the  Aid  in  a simulated  threat  situation  and  provide  his 
judgments  about  the  Aid  by  responding  to  the  questionnaire. 
These  judgments  would  provide  an  assessment  of  the  Aid's 
conceptual  value  as  well  as  an  indication  of  the  Aid's  per- 
ceived strengths  and  weaknesses  as  a basis  for  software 
refinement. 


2.3.2  Description  of  pilot  test  procedure 


Subjects:  Eight  subjects  were  used  in  the  work- 

shop trials.  These  subjects,  chosen  from  the  following  six 
organizations,  represented  a wide  range  of  naval  interests 
and  experiences: 


1.  Office  of  the  Chief  of  Naval  Operations  (OP-05) 

2.  Office  of  the  Chief  of  Naval  Operations  (OP-090) 

3.  Office  of  the  Chief  of  Naval  Operations  (OP-098) 

4.  Naval  Intelligence  Command  (NFOIO) 

5.  Office  of  Naval  Research 

6.  Commander  in  Chief  Atlantic  Fleet 

Test  Plan:  The  test  plan  was  designed  to  pro- 

vide the  subject  with  as  thorough  an  understanding  of  the 
Aid's  function,  properties,  and  features  as  possible  within 
the  time  available  for  evaluation  (typically  three  to  four 
hours) . This  familiarization  process  enabled  the  subjects 
to  give  reasonably  informed  judgments  in  response  to  the 
questionnaire.  As  mentioned  previously,  no  attempt  was  made 
to  perform  a controlled  experiment  for  aid  evaluation. 
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Six  distinct  steps  ware  contained  in  tha  testing 
procedure.  The  initial  two  steps  took  the  form  of  a briefing 
and  served  to  acquaint  each  subject  with  the  overall  objec- 
tive of  the  evaluation.  First,  the  subject  was  introduced 
to  the  purpose  of  the  test  and  the  ONR  Operational  Decision 
Aids  (ODA)  project.  In  the  briefing,  we  explained  the 
history  of  the  ODA  project,  DDI's  previous  and  current  task 
orientation  in  the  project,  the  outline  of  the  test  that 
would  follow,  and  the  general  form  in  which  the  subject's 
responses  would  be  recorded.  Second,  we  introduced  the 
subject  to  the  tactical  situation  facing  the  task  force 
commander — an  "enemy"  ASM  threat  in  the  ONRODA  scenario — 
using  a sequence  of  graphic  slides. 

After  this  introduction,  the  subject  was  exposed 
to  the  Aid  in  a series  of  off-line  (vu-graph)  and  on-line 
(computer- graphic)  interactions,  and  his  responses  were 
recorded.  In  the  third  step  of  the  test,  the  subject  was 
exposed  to  the  ASM  threat  scenario  with  an  off-line  descrip- 
tion of  Red's  possible  intents,  possible  actions  that  the 
subject  might  take  in  response  to  the  threat,  and  the  indi- 
cators of  threat  activity  that  would  be  received  as  the 
threat  situation  developed.8  Next,  the  subject  was  exposed 
to  a representative  sequence  of  threat  indicators  in  an  on- 
line interaction  with  the  computerized  Aid.  In  the  fourth 
step  of  the  test,  the  subjects  were  introduced  to  the  proba- 
bilistic component  of  the  Aid.  The  Aid's  probability  plane 
was  explained  off-line  to  the  subject  using  two  examples. 
Then,  the  sequence  of  indicators  used  in  step  three  was 
presented  to  the  subjects  on-line,  but  now  a probabilistic 
interpretation  of  the  data,  derived  from  the  Bayesian  proba- 
bility model  explained  in  Section  2.2.2  above,  was  also 
shown.  In  the  fifth  step  of  the  test,  the  subject  was 
introduced  to  the  value  threshold  component  of  the  Aid.  A 
series  of  off-line  illustrations  were  used  to  explain  the 
meaning  of  the  Aid's  value  matrices  and  how  these  matrices 
combined  to  yield  probability  thresholds.  Then,  the  same 
sequence  of  indicators  was  presented  to  the  subject,  and  the 
probabilistic  and  value  information  was  used  to  provide  a 
display  of  the  preferred  decision.  At  this  point,  the 
subject  was  working  with  all  components  of  the  Aid,  and  he 
was  encouraged  to  interact  with  the  Aid,  exercising  its 
various  features  and  decision  analytic  properties. 


8The  vu-graphs  used  in  the  off-line  presentations  are  re- 
produced in  Appendix  B of  this  report. 
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In  the  sixth  and  final  step  of  the  test,  the 
subject  was  debriefed  and  asked  to  respond  to  the  items  in 
the  questionnaire,  and  his  judgments  were  recorded.  The 
entire  procedure  took  approximately  three  to  four  hours  to 
complete. 

2.3.3  Findings  and  recommendations  - The  subjects' 
evaluations  and  judgments  expressed  during  the  pilot  test 
indicate  that  there  is  sufficient  conceptual  acceptance  of 
the  Execution  Aid  to  justify  its  continued  development  as  a 
tactical  decision  aid.  The  subjects'  responses  also  indicate 
that  the  Aid  may  be  valuable  as  a training  device,  to  enable 
new  task  force  commanders  to  become  familiar  with  the  threat 
and  to  test  out  alternative  response  actions  in  a simulated 
environment.  The  subjects'  responses  also  indicate  that  the 
Aid  might  possibly  be  useful  at  other  command  levels  such  as 
the  numbered  fleet  commander. 

Additionally,  numerous  suggestions  were  made  for 
refining  the  computer  software  to  increase  the  Aid's  useful- 
ness and  comprehension.  These  suggestions  include: 

1.  A capability  to  recall  and  display  an  action/ 
response  option  checklist  automatically  as  the 
"probability  bug"  crosses  a threshold  into 
another  decision  area. 

2.  A provision  that  allows  unanticipated  diagnostic 
data  to  be  incorporated  into  the  model  for  sup- 
plementing the  indicator  list. 

3.  A way  to  incorporate  an  indication  of  time 
remaining  to  undertake  a response  option  into 
the  analysis. 

4.  A summary  of  indicators  that  have  been  received 
by  the  command  center  on  the  "action  indicator" 
display  for  the  commander's  use. 

5.  A display  of  the  expected  value  of  the  actions. 

6.  Automatic  scrolling  of  the  indicator  list  so 
that  the  indicator  received  last  will  be  shown. 

7.  A way  to  handle  more  than  three  states. 

The  computer  software  changes  responsive  to  these  suggestions 
as  well  as  other  software  changes,  for  example,  a front-end 
prompt  and  editing  feature  and  an  expanded  indicator  list 
(which  includes  all  of  the  latest  air/submarine  threat  indi- 
cators) , are  described  in  the  user's  manual  currently  under 
development. 
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This  testing  activity  has  sufficiently  identified 
refinements  to  justify  a future  controlled  evaluative  experi- 
ment. Such  an  experiment  would  attempt  to  measure  a subject's 
performance  in  a simulated  situation  both  with  and  without 
the  Aid.  Section  4.3  below  recommends  the  prosecution  of  am 
experiment  of  this  nature  as  a future  task. 
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3.0  INVESTIGATION  OF  AID  GENERALIZATIONS 


In  principle , the  Execution  Aid  combines  the  method- 
ologies of  multi-attribute  utility  theory  and  Bayesian 
probability  updating.  The  initial  software  prototype  Ex- 
ecution Aid,  however,  represented  a limited  development  of 
the  potential  power  of  the  methodological  techniques  as  they 
were  implemented  in  a computer-graphic  form.  This  was  due, 
in  part,  to  the  display  initially  chosen,  and  in  part  to  the 
degree  of  investigation  possible  under  the  previous  contract 
phase.  Thus,  in  order  to  enhance  the  potential  of  the  Aid, 
we  investigated  several  ways  to  achieve  a more  comprehensive 
Aid.  Accordingly,  our  technical  proposal  outlined  two  areas 
that  required  further  investigation,  a method  to  model  more 
them  three  state  hypotheses  and  a method  to  explicitly 
incorporate  time  into  the  analysis. 1 in  addition,  our 
investigation  under  the  current  contract  phase  indicated 
that  it  may  be  important  to  incorporate  the  decision  maker's 
risk  attitude  in  the  analysis.  The  final  investigation 
reported  below,  to  search  for  more  generalized  uses  of  the 
Aid  in  a variety  of  tactical  situations,  was  undertaken  at 
the  request  of  the  Scientific  Officer.  The  following  sections 
present  our  investigations  of  these  areas.2 

3. 1 More  Than  Three  State  Hypotheses 

Due  to  the  geometry  utilized  in  the  current  Execution 
Aid,  only  three  state  hypotheses  can  be  considered  in  our 
decision  modeling.  That  is,  a two-dimensional  triangular 
display  can  completely  represent,  at  most,  three  state 
hypotheses.  This  fact  is  illustrated  in  Figure  3-1.  The 
top  figure  shows  the  space  of  three  mutually  exclusive  and 
collectively  exhaustive  states  1,  2,  and  3.  According  to 
the  rules  of  probability  theory,  all  admissible  probability 
combinations  must  satisfy  the  following  conditions: 

p(l)  + p (2)  + p (3)  = 1.0 

0.0  < p (1)  ^ 1.0 

0.0  < p (2)  ^ 1.0 

0.0  < p (3)  ^ 1.0 

"Proposal  for  Follow-on  Research  and  Development  on  Decision 
Analysis  as  an  Element  in  an  Operational  Decision  Aiding  System 
(Phase  III)"  for  Office  of  Naval  Research,  3 October  1975,  pp.4-5. 

2 

The  following  sections  contain  fairly  detailed  technical 
material;  a reader  who  is  unfamiliar  with  the  concepts  of  the 
Execution  Aid  should  review  section  3 of  Brown  et  al.  (1975). 
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It  can  be  readily  seen  that  these  conditions  specify  the 
probability  triangle  shown  at  the  bottom  of  Figure  3-*l. 

Thus,  the  three-state  probability  space  can  be  completely 
represented  in  two  dimensions  by  the  triangle,  and  the  tri- 
angle is  the  display  used  in  the  prototype  Execution  Aid. 

The  other  feature  of  the  Aid  is  that  of  probability 
thresholds  for  actions.  Figure  3-2  illustrates  the  decision 
thresholds  for  the  actions  that  might  be  taken  in  response 
to  the  three  states.  The  top  of  the  figure  shows  a table  of 
assessed  values,  representing  the  amount  of  loss  associated 
with  each  action  for  each  state  of  Red  intent.  Since  the 
action  with  the  highest  expected  value  is  always  preferred, 
thresholds  can  be  calculated  for  the  actions  in  terms  of  the 
three  probabilities.  Since  the  thresholds  are  planes  in  the 
three  dimensional  state-space,  they  intersect  the  probability 
triangle  in  straight  lines.  Thus,  the  probability  triangle 
can  be  divided  into  action  areas,  shown  as  counter-measure 
areas  at  the  bottom  of  Figure  3-2.  The  location,  or  point 
of  probability  of  the  prevailing  states,  will  then  yield  the 
preferred  counter-measure  decision.  Figure  3-2  illustrates 
the  example  of  a decision  to  take  no  counter-measure  as  the 

preferred  action  when  p(l)  = .4,  p(2)  = .2,  and  p(3)  * .4. 3 

The  triangular  display  is  sufficient  for  those  situations 
where  only  three  states  are  important.  In  general,  though, 
this  may  not  be  the  case,  and  this  characteristic  has  been 
brought  to  our  attention  by  the  test  subjects  and  others. 

Thus,  there  exists  a need  to  find  a way  of  providing  a . 

sufficient  description  of  a model  of  more  than  three  states. 

Several  alternatives  for  handling  a state  space  with 
more  than  three  hypotheses  were  investigated.  These  methods 
include: 

o structuring  the  problem  in  a hierarchical  manner 
in  which  each  level  of  the  hierarchy  has  three 
states; 

o displaying  the  distances  to  the  thresholds  instead 
of  a complete  representation  of  the  state  space? 


A fuller  description  of  the  probability  and  threshold 
displays  is  presented  in  Brown  et  al.  (1975),  pages  3-6 
through  3-19. 

4 

Increasing  the  number  of  states  to  more  than  three  is 
equivalent  to  increasing  the  number  of  branches  on  the 
underlying  probability  node  to  more  than  three. 
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o projecting  thresholds  and  probabilities  onto 
planes  in  the  state  space;  and 

o displaying  cutting-planes  of  the  state  space. 

3.1.1  Status  and  conclusions  - The  investigations  into 
graphic  display  modeling  techniques  for  the  treatment  of 
more  than  three  state  hypotheses  while  not  conclusive  appear 
adequate  for  this  stage  of  the  development  of  decision 
analytic  prototype  decision  aids.  The  central  question 
remaining  is  the  frequency  and  severity  of  situations  that 
will  generate  more  than  three  state  hypotheses  of  concern, 
and  the  requirement  for  a corresponding  degree  of  graphic 
sophistication  required  to  cope  with  these  instances.  Of 
the  various  methods  investigated,  we  cannot  favor  any  par- 
ticular technique  as  the  most  versatile  with  a high  probability 
of  meeting  the  major  uncertainties  of  interest  to  a tactical 
commander.  We  believe  however,  that  further  investigation 
should  be  constrained  by  the  outcome  of  the  planned  evalua- 
tion phase  during  which  user  interaction  in  a testbed 
environment  will  identify  the  strengths  and  weaknesses  of 
prototype  decision  aids.  (It  is  noted  that  Analytics, 
Incorporated,  is  investigating  the  use  of  nomographs  for 
displaying  tactical  data  for  action  selection.) 

3.2  Explicit  Consideration  of  Time 

For  many  situations,  it  appears  that  when  to  take  an 
action  is  as  difficult  a decision  as  what  action  to  take. 

Thus,  it  is  important  to  investigate  ways  in  which  the  time 
factor  can  be  considered  as  a feature  of  the  Execution  Aid. 

Our  investigation  to  date  has  focused  on  four  alternatives 
for  addressing  time,  all  of  which  were  examined  within  the 
context  of  the  air/submarine  ASM  threat  described  in  Section 

2.2.2  above.  These  alternatives  were: 

1.  Time  windows  for  the  possible  response  actions. 

2.  Time  projections  of  the  probability  bug. 

3.  Time- dependent  value  functions. 

4.  Time/date  displays  for  indicators  received. 

3.2.1  Time  windows  - In  the  time-window  method,  the 
commander's  display  of  the  Execution  Aid  would  provide 
information  on  the  time  remaining  to  begin  each  action  in 
order  to  be  assured  of  enough  time  to  complete  it.  Table 
3-1  shows  the  estimates  that  are  needed  to  calculate  each 
time  window.  If  it  takes  the  task  force  an  amount  of  time 

Tc  to  complete  an  action,  for  example  to  "prepare  to  attack," 
then  the  action  must  be  started  Tc  before  tA,  the  time  at 
which  the  enemy  attacks.  Thus,  at  any  time  to,  the  time 
window  for  the  "prepare  to  attack"  action,  the  amount  of 
time  that  remains  before  the  commander  must  order  the  action 
to  begin,  is  equal  to:  tA  - Tc  - tQ. 
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Table  3 1:  TIME  WINDOW 


The  current  time,  to,  is  trivial  to  determine 
and  requires  no  aid  other  than  a ship's  clock.  The  time  to 
complete  an  action,  Tc,  can  be  estimated  fairly  accurately, 
although  not  precisely,  in  advance  and  should  be  provided 
for  in  the  aid.  However,  it  appears  virtually  impossible  to 
predict,  in  advance,  the  time  of  enemy  attack,  tA,  with 
enough  accuracy  to  be  useful  in  the  analysis. 

In  the  air/submarine  threat,  for  example,  it  is 
possible  for  the  enemy  to  maintain  a full  readiness  to 
attack  posture  for  48  hours  or  more.  Thus,  an  attack  time 
that  is  estimated  in  advance  and  conditioned  on  anticipated 
future  activities  cannot  be  accurate  to  within  less  than  48 
hours.  Thus,  a window  cannot  be  estimated  to  within  less 
than  48  hours,  and  such  an  estimate  is  of  little  use  in  the 
dynamic  situation  for  which  the  Aid  was  designed.  Therefore, 
the  time-window  method  appears,  at  the  present  time,  to  be 
an  unsatisfactory  method  of  incorporating  an  explicit  con- 
sideration of  time  into  the  Aid. 

3.2.2  Time  projections  of  the  probability  bug  - In  the 
time-projection  method,  the  position  of  the  probability  bug 
is  projected  based  upon  an  estimate  of  the  timing  between 
indicators,  for  example  as  shown  in  Figure  3-3.  In  this 
example,  the  current  position  is  the  moment  that  a report  is 
received  of  Red  withdrawing  his  surface  ships,  which  yields 
probabilities  of  .75,  .05,  and  .20  for  the  states  routine, 
feint,  and  attack,  respectively.  In  the  probability  pro- 
jection method,  the  time  between  the  receipt  of  indicators 
is  estimated  in  advance.  In  this  example,  a time  span  of 
four  hours  is  estimated  for  the  time  between  the  receipt  of 
a report  of  Red's  surface  activity  and  the  receipt  of  a 
report  of  his  communication  activity.  In  addition,  a report 
of  Red's  submarine  activity  is  assumed  to  occur  four  hours 
after  the  receipt  of  a report  of  his  communication  activity. 

Based  on  the  current  location  of  the  probability 
bug  and  the  likelihoods  associated  with  the  possible  commu- 
j nication  activities,  in  four  hours  the  probability  bug  will 

be  at  some  location  along  the  dotted  line  as  shown  in  Figure 
3-3.  Considering  the  likelihoods  associated  with  the 
* possible  submarine  activities,  in  eight  hours  the  probability 

bug  will  be  at  some  location  within  the  area  bounded  by  the 
beaded  line,  as  shown.  In  this  illustration,  the  time 
projection  of  the  probability  bug  indicates  that  the  "routine" 
I action  will  continue  to  be  preferred  in  eight  hours,  because 

the  eight-hour  projection  lies  completely  within  the  "Blue 
routine"  action  area. 

Notice  that  the  probability  projection  method 
requires  a prediction,  in  advance,  of  both  the  ordering  of 
the  receipt  of  indicators  and  the  timing  between  receipts. 


These  requirements  dictate  an  unrealistically  strong  assump- 
tion of  the  predictability  of  the  enemy  threat  and  the 
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ability  to  process  information.  In  particular,  wo  must 
assume  that  the  enemy's  activities  will  always  occur  in  the 
same  order  and  with  the  same  timing  between  activities,  that 
we  will  detect  each  activity  at  a definite  time  after  it 
occurs,  and  that  the  report  of  an  activity  will  always  take 
the  same  amount  of  t,ime  to  process.  In  addition,  all  of 
these  quantities -must  be  determined  in  advance. 

We  feel  that  these  assumptions  are  too  unrealis- 
tic for  any  real  event  situation  and  that  the  time  projection 
feature  of  the  probability  buq,  while  it  is  an  interesting 
idea,  is  not  a practical  method  for  incorporating  time  into 
the  analysis. 


3.2.3  Time-dependent  value  functions  - The  time- 
dependent  value  function  method  requires  the  commander  to 
assess  several  conditional  value  tables,  which  are  used  in 
the  analysis,  as  appropriate.  For  example,  one  value  table 
might  be  elicited  under  the  condition,  "if  Red  is  going  to 
attack,  the  attack  will  occur  in  48  hours,"  and  another 
value  table  might  be  elicited  under  the  condition,  "if  Red 
is  going  to  attack,  the  attack  will  occur  in  8 hours."  If 
it  is  less  costly  to  take  preparatory  actions  when  Red’s 
possible  attack  will  occur  in  a short  time  period,  then  the 
probability  thresholds  at  which  such  actions  are  indicated 
are  lower,  for  instance,  as  shown  in  Figure  3-4.  In  the 
Aid's  operation,  the  "48-hour"  value  table  is  used  to  deter- 
mine the  decision  thresholds  when  this  is  assessed  time 
until  attack,  and  the  "8-hour"  value  table  is  used  to  deter- 
mine the  thresholds  when  the  assessed  time  until  attack  is  8 
hours.  Notice  that  this  use  of  time-dependent  value  functions 
requires  a prediction  of  the  time  at  which  Red  will  attack. 
Thus,  this  use  of  the  time-dependent  value  method  is  unsatis- 
factory for  the  same  reasons  that  the  time-window  method  is, 
as  explained  in  Section  3.2.1  above. 

Although  it  appears  that  the  time  at  which  Red 
will  attack  cannot  be  predicted  with  precision,  the  minimum 
time  to  attack,  determined  from  Red's  capability  to  attack, 
can  be  predicted  with  enough  accuracy  to  incorporate  its  use 
in  the  analysis.  That  is,  a sequence  of  indicators  could  be 
hypothesized  to  predict  the  amount  of  time  that  would  be 
necessary  for  Red  to  become  capable  of  attack.  However, 
since  the  enemy  could  attain  a full  readiness  capability 
within  an  hour,  if  the  launch  and  targeting  platforms  are 
within  range,  and  since  the  enemy  could  hold  the  full  readi- 
ness posture  for  48  hours  or  more,  the  assessed  value  tables 
will  not  differ  significantly  over  the  relevant  range  of 
capability  to  attack  (capable  within  one  hour  or  less) . 

That  is,  even  if  an  estimate  of  capability  to  attack  is  used 
to  predict  Red's  timing,  the  relevant  time  for  purposes  of 
evaluating  the  action  alternatives  is  still  the  attack  time. 

By  introducing  the  capability  time,  the  consideration  of  the 


attack  time  has  changed  from  a consideration  of  a point  to  a 
consideration  of  a probability  distribution.  That  is,  for 
purposes  of  determining  the  value  of  the  action,  the  time  at 
which  the  attack  will  occur  is  relevant  if,  as  in  this 
example,  the  enemy  can  hold  a fully  capable  posture  for  48 
hours.  Then  the  values  assessed  for  actions  given  that  the 
enemy  is  fully  capable  must  implicitly  consider  a proba- 
bility distribution  for  attack  time  spanning  the  period  zero 
through  48  hours.  Similarly,  the  values  assessed  for  actions 
given  that  the  enemy  will  be  fully  capable  in  one  hour  must 
implicitly  consider  a probability  distribution  for  attack 
time  spanning  the  period  from  one  through  49  hours.  An 
examination  of  Figure  3-5  shows  that  the  distributions  of 
attack  times  will  not  be  significantly  different  and,  thus, 
the  value  assessments  are  not  expected  to  vary  signifi- 
cantly. If  the  value  assessments  are  not  significantly 
different,  then  the  effort  required  to  make  the  assessments 
does  not  appear  justified. 

Therefore,  it  appears  that  the  time-dependent 
value  function  method  is  unpromising  for  the  ASM  threat  and 
other  tactical  situations  where  the  enemy  can  attain  full 
capability  relatively  quickly  and  hold  that  capability  for 
a relatively  long  period  of  time. 

3.2.4  Time/date  display  - The  first  three  investiga- 
tions of  time  appeared  to  hinge  on  predictions  that  are 
nearly  impossible  to  usefully  make  in  advance,  and  the 
evidence  considered  so  far  indicates  that  the  explicit 
incorporation  of  time  into  the  Execution  Aid  is  an  unprom- 
ising area  of  investigation.  Still,  it  was  felt  that  some 
modifications  could  be  made  to  provide  the  decision-maker 
with  improved  time-related  information  so  that  he  could  do 
a better  job  of  considering  time  implicitly  in  his  decision 
process.  The  first  provision  involves  an  estimate  and 
display  of  the  expected  time  required  to  complete  each 
action.  (This  is  Tc  as  explained  in  Section  3.2.1  above.) 
Another  provision  involves  identifying  the  time  and  date  at 
which  the  indicators  are  received.  Although  these  con- 
cessions to  the  time  problem  are  minimal,  they  are  the  only 
ones  that  can  be  recommended  at  this  time. 

3.2.5  Status  and  conclusions  - The  minimal  time  pro- 
visions, date/time  tags  on  the  indicators  and  time-to- 
complete  tags  on  the  actions,  have  been  incorporated  into 
the  Execution  Aid  software. 

Although  the  explicit  consideration  of  time 
initially  appeared  to  be  a promising  area  for  aid  refine- 
ment, a further  investigation  of  some  of  the  specific  ways 
that  time  could  be  addressed  proved  to  be  of  questionable 
value.  In  particular,  the  methods  of  using  time  windows. 


time  projections  of  probabilities,  and  time-dependent  value 
functions  involved  estimates  that  were  considered  to  be 
virtually  impossible  to  make  in  advance  with  the  accuracy 
required  for  action  selection. 

These  investigations  lead  us  to  conclude  tenta- 
tively that  an  explicit  incorporation  of  time  into  the 
Execution  Aid  is  not  a promising  area  for  aid  refinement. 
Further  investigation  is  necessary,  however,  to  make  a 
definitive  statement  on  this  issue. 

3.3  Consideration  of  Risk5 


An  investigation  into  the  sensitivity  of  the  action 
thresholds  to  the  decision-maker's  attitude  toward  risk 
revealed  that  even  slight  risk  aversion  can  cause  signifi- 
cant changes  in  the  thresholds.  However,  since  an  investi- 
gation into  the  incorporation  of  risk  as  a property  of  the 
Execution  Aid  was  an  extension  of  the  proposed  effort,  the 
investigation  has  been  limited  at  this  time.  We  have, 
however,  briefly  reviewed  several  alternatives  for  addressing 
risk;  but  we  have  not  as  yet-  concluded  which  method,  if  any, 
is  best.  (The  Naval  Personnel  Research  and  Development  Center 
is  completing  an  investigation  of  risk  as  a factor  for  in- 
corporation in  candidate  decision  aids.) 

3.3.1  Problem  and  need  specification  - The  value 
tables  as  presently  developed  In  the  Execution  Aid  were 
assessed  under  the  condition  of  certainty;  that  is,  the 
value  for  an  action  was  assessed  assuming  that  Red's  intent 
is  known  for  certain.  Thus,  the  thresholds  are  calculated 
on  an  expected  value  basis,  which  assumes  that  the  decision- 
maker is  "risk- neutral. " Since  decision-makers  do  not 
always  exhibit  this  attitude  toward  risk,  but  rather  an 
attitude  which  is  risk-seeking  or  risk-averse,  the  sensitivity 
of  the  thresholds  to  the  assumption  of  risk  neutrality 
deserves  further  investigation. 

For  purposes  of  checking  the  sensitivity  of  the 
thresholds  to  risk,  an  experimental  function  of  the  form 
u(x)  = £ (1  - e"rx) , where  r is  the  coefficient  of  risk 

? c 

aversion  and  c is  a scaling  constant  , was  used  to  calculate 


Some  of  the  material  in  this  section  assumes  that  the  reader 
has  a technical  knowledge  of  the  concepts  of  risk  attitude 
and  utility  theory.  Introductory  discussions  of  these  topics 
appear  in  Chapters  4 and  18  of  Brown,  Kahr,  and  Peterson 
(1974);  Chapter  4 of  Raiffa  (1968);  Chapter  5 of  Schlaifer 
(1969);  and  Chapter  5 of  Handbook  for  Decision  Analysis  (1973) 


’This  is  a positive,  linear  transform  of  the  function  of  con- 
stant risk  aversion  described  on  page  156  of  Schlaifer  (1971) . 
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the  utility  for  the  value* , u(x).  This  function  wa*  chosen 
because  it  exhibits  a constant  aversion  to  risk,  which  is  a 
reasonable  approximation  of  a decision  maker's  actual  risk 
aversion  in  many  cases,  and  because  it  facilitated  calcula- 
tions.7 For  the  purpose  of  checking  the  sensitivity  of  the 
thresholds  to  risk,  the  curve  u(x)  - 303  (1  - e-x/250)  was 
chosen  because  it  does  not  differ  significantly  from  a risk 
neutral  curve,  as  shown  in  Figure  3-6.  If  the  analysis  is 
insensitive  to  risk,  then  the  thresholds  derived  by  using 
this  utility  curve  should  be  virtually  the  same  as  the 
thresholds  derived  by  using  the  risk  neutral,  expected  value 
curve. 


The  example  in  Figure  3-7  shows,  however,  that 
this  is  not  the  case.8  Introducing  risk  into  the  analysis 
causes  the  areas  for  actions  "H"  and  "R"  to  collapse  to 
about  15%  and  60%  of  their  riskless  sizes,  respectively, 
while  the  areas  for  actions  "P"  and  "S"  increase  by  about 
25%  and  85%  respectively.  Clearly,  such  changes  are  sig- 
nificant, indicating  that  it  is  worthwhile  to  investigate 
methods  of  incorporating  risk  into  the  analysis  to  assure 
maximum  usefulness  in  the  Aid's  configuration. 

3.3.2  Alternative  approaches  - Decision  analysis 
literature  offers  several  alternative  methods  for  incorpo- 
rating risk  into  the  analysis.  However,  it  is  not  clear 
what  method,  if  any,  can  be  usefully  incorporated  into  the 
prototype  Execution  Aid  in  its  present  use  and  form. 

One  method  for  handling  risk  involves  assuming 
the  form  of  the  utility  function,  such  as  an  exponential, 
and  then  eliciting  the  necessary  parameters  from  the  decision 
maker.  For  instance,  in  the  case  of  the  exponential  utility 
function  described  in  the  previous  section,  the  value  of  the 
parameter  r would  be  elicited  from  the  decision  maker.  This 
technique  is  available  in  the  computer  program  described  in 
Zamora  and  Leaf  (1974) . This  method,  however,  requires  the 
decision-maker  to  comprehend  both  the  implication  of  the 
functional  form  chosen  and  the  curve  that  would  result  from 
using  any  particular  value  of  the  parameters.  These  assump- 
tions are  not  easily  met  in  practice  by  naval  personnel 
without  extensive  training,  so  this  method  would  probably 
not  be  a useful  feature  of  the  Execution  Aid. 


7 Pratt  (1964)  contains  a full  discussion  of  the  measure  of 
risk  aversion  and  examples  of  functions  exhibiting  constant, 
increasing,  decreasing,  and  constraint-proportional  risk  aversion. 

8The  expected  value  (risk  neutral)  results  of  tnis  example 
are  approximately  the  same  as  those  obtained  from  our  model 
of  the  air/submarine  ASM  threat. 
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Figure  3-6 

RISK  AVERSE  AND  RISK  NEUTRAL  UTILITY  CURVES 


Another  method  to  handle  risk  assumes  the  func- 
tional form  of  the  utility  curve,  but  does  not  require  the 
decision-maker  to  enter  parameters.  Instead,  he  specifies 
his  risk  attitude  in  a reference  gamble.  A curve,  which 
exhibits  the  specified  form,  is  then  fit  to  the  information 
contained  in  the  reference  gamble.  This  reference-gamble 
method,  available  in  the  computer  program  described  in 
Schlaifer  (1971),  does  not  require  the  decision-maker  to 
know  what  curve  will  result  from  a particular  choice  of  the 
parameters;  but  it  still  requires  him  to  know  the  implica- 
tions of  the  functional  form  chosen.  Thus,  the  reference- 
gamble  approach  may  be  too  demanding  of  the  decision-maker 
to  be  routinely  useful  in  the  Execution  Aid. 

As  another  alternative,  the  utility  table  could 
be  evaluated  directly  by  eliciting  values  based  upon  lot- 
teries for  different  act/state  combinations  using  a method 
such  as  the  7T-BRLT  procedure  described  in  Chapter  4 of 
Raiffa  (1968) . Determining  a utility  table  directly,  how- 
ever, is  probably  too  laborious  and  time-consuming  (since 
many  lotteries  must  be  considered)  to  be  useful  in  the 
Execution  Aid's  implementation. 

3.3.3  Conclusions  and  findings  - Our  limited  investi- 
gations to  date  indicate  that  is  is  important  to  find  a way 
in  which  the  decision-maker's  attitude  toward  risk  can  be 
usefully  incorporated  into  the  Execution  Aid.  Furthermore, 
it  appears  that  traditional  methods  of  addressing  risk  may 
be  so  difficult  to  apply  in  practice  that  they  reduce  rather 
than  enhance  the  usefulness  of  the  Aid  by  making  it  too 
cumbersome  to  operate.  Thus,  new  approaches  to  the  problem 
of  modeling  risk  attitudes  may  need  to  be  developed,  and 
this  area  is  recommended  for  future  investigation. 

3.4  Generalized  Use 

At  the  direction  of  the  Scientific  Officer,  we  investi- 
gated the  generalized  use  of  the  prototype  Execution  Aid  in 
situations  other  than  a task  force's  response  to  the  ASM 
threat.  This  investigation  has  taken  two  directions.  First, 
the  properties  of  the  Aid  were  examined  with  the  view  that 
the  key  property  of  this  aid  is  one  of  probability  thresholds 
and  that  this  property  is  most  applicable  in  situations 
where  states  are  probabilistically  independent  of  actions. 
Based  on  this,  the  ONRODA  scenario  was  re-examined,  and  the 
strike  timing  decision  was  identified  as  one  that  meets  this 
condition  with  the  reservation  that  more  than  one  tier  of 
conditioning  events  must  be  considered.  Section  3.4.1  below 
describes  the  application  of  the  Execution  Aid  to  the  strike 
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timing  decision.  Second,  in  addition  to  this  generalized 
application,  the  basic  properties  of  decision  analysis  (that 
is,  probability  and  utility  theory)  were  applied  in  a different 
manner  to  the  problem  of  target  selection  in  the  "Amphibious 
Warfare  Scenario"  (Rowney  [1975] ) . This  application  is 
explained  in  section  3.4.2  below. 

3.4.1  Execution  aid-contingent  mode  - The  main  finding 
from  the  investigation  of  the  generalized  use  of  the  Execution 
Aid  was  an  identification  of  the  key  situationally  dependent 
property  of  the  Aid,  that  of  probability  thresholds.  That 
is,  the  Aid  is  designed  to  monitor  the  environment  and 
recommend  a course  of  action  based  upon  probability  thresh- 
olds. For  this  reason,  the  Aid  seems  most  suited  to  addressing 
operational  contingencies,  events  of  possible  but  uncertain 
occurrence,  that  would  be  disruptive  enough  to  cause  a 


change  in  the  mission  plan.  In  addition,  it  appears  that 
the  Aid  is  most  useful  when  the  problem  can  be  structured  so 
that: 

1.  The  state  probabilities  are  independent  of  the 
actions;  and 

2.  The  value  of  the  actions  can  be  assessed  as  a 
function  of  the  states  (that  is,  the  specification 
of  an  action/state  pair  is  a sufficient  description 
of  the  situation  to  assign  a value) . 

For  such  problems  it  is  proper  to  consider  the  appropriate- 
ness of  the  actions  in  terms  of  the  probabilities  of  the 


states  by  establishing  probability  thresholds.  Thus,  it  is 
proper  to  think  of  the  Execution  Aid  as  a Contingecy  Aid  and 
to  seek  out  contingencies  that  exhibit  the  two  features 
described  above. 

One  such  contingency  that  we  feel  meets  the 
conditions  of  probabilistic  independence  of  actions  and 
states  and  functional  dependence  of  an  action's  value  on  the 
state  is  that  of  strike  mission  timing,  whether  to  change 
the  planned  time  of  a strike.  This  decision  situation  has 
been  examined  within  the  context  of  the  air  strike  mission 
in  the  ONRODA  scenario  and  is  illustrated  below. 

In  the  operational  situation  presented,  an  early 
morning  air  strike  is  planned  against  the  airfield  on  ONRODA 
island.  However,  when  planning  for  this  mission,  the  task 
force  commander  (CTF)  recognized  the  possibility  that  the 
mission  may,  at  the  last  minute,  have  to  be  delayed  due  to 
unfavorable  weather,  the  state  of  readiness  of  his  own 
forces,  the  state  of  readiness  of  the  enemy,  or  some  combi- 
nation of  these  factors.  In  the  face  of  these  uncertainties. 
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delay  of  the  initial  strike  for  either  an  hour  or  a day  was 
considered.  A decision  tree  structure  of  this  decision 
problem  is  shown  in  Figure  3-8.  Here,  the  action  node  is 
followed  by  three  tiers  of  conditioning  events,  one  each  for 
the  states  of  weather,  enemy  readiness,  and  own-force  readi- 
ness. Each  tier  of  events  is  characterized  by  three  state 
possibilities.  Weather  can  be  classified  as  good,  marginal, 
or  bad;  enemy  readiness  can  be  classified  as  unprepared, 
partially  prepared,  or  completely  prepared;  and  own-force 
readiness  can  be  classified  as  completely  prepared,  partially 
prepared,  or  unprepared.  Thus,  there  are  twenty-seven 
combinations  of  event  states  that  must  be  considered  in 
connection  with  each  action. 

An  evaluation  of  this  decision  situation  indi- 
cated that  the  strike-timing  problem  could  be  adequately 
modeled  by  treating  each  event  as  probabilistically  inde- 
pendent of  the  actions  and  of  the  other  events,  and  by 
assessing  values  based  directly  on  the  action/state  com- 
binations. Thus,  this  problem  satisfied  the  contingent 
conditions  that  are  necessary  to  apply  the  Execution  Aid. 

However,  it  was  recognized  that  the  display  of 
the  Execution  Aid,  a single  probability  triangle,  could 
handle  only  one  tier  of  conditioning  events  and  that  the 
strike-timing  decision  problem  required  three  tiers  of 
conditioning  events.  A method  had  to  be  found  to  incor- 
porate the  additional  tiers.  Based  upon  our  earlier 
investigation  of  the  generalization  to  consider  more 
than  three  states  within  a single  tier  (described  in 
Appendix  0) , we  recognized  that  a display  of  multiple  tri- 
angles, in  the  cutting-plane  method,  offered  an  appealing 
and  manageable  display.  Investigation  revealed  that  a 
multiple-triangle  display  could  be  developed  for  several 
conditioning  tiers  in  a manner  analogous  to  the  cutting- 
plane  method  used  for  more  them  three  states  within  a tier. 


Figure  3-9  shows  the  multiple-triangle  graphic 
display  for  the  strike-timing  decision.  Given  the  initial 
probability  assessments  for  the  three  state  variables,  the 
preferred  action  is  to  proceed  with  the  strike  as  planned, 
with  no  delay.  The  display  shows  that  at  the  initial  prob- 
ability assessment,  this  decision  is  most  sensitive  to 
changes  in  the  weather;  that  is,  the  weather  probability  bug 
is  close  to  the  decision  threshold  to  delay  an  hour.  The 
decision  is  insensitive  to  the  state  of  enemy  readiness  (at 
the  assessed  state  of  own-force  readiness  and  weather) , and 
no  change  in  the  assessment  of  the  enemy's  readiness  would 
be  reason  enough  to  delay  the  strike.  Finally,  the  decision 
is  somewhat  sensitive  to  the  state  of  own-force  readiness, 
where  a significant  change  in  this  assessment  would  be 
reason  enough  to  delay  the  strike. 
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Figure  3-8 

STRIKE  TIMING  DECISION  - TREE  STRUCTURE 


STRIKE  TIMING  DECISION  - GRAPHIC  DISPLAY 
(INITIAL  PROBABILITY  ASSESSMENT) 
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The  decision  thresholds  «sr«  determined  from  ths 
assessed  value  tables  shown  in  Table  3-2.  Xe  this  esse,  the 
values  of  the  action/weather  cost) inat ions  were  assessed  to 
be  independent  of  the  other  events,  however,  the  ow-foroe 
readiness  and  enemy  readiness  states  needed  to  he  ooesidered 
in  combination  for  valuation  purposes.  The  import  sene 
weights  show  that  ths  readiness  of  foroes  has  twice  as  meeh 
effect  on  the  strike-timing  decision  as  the  weether  dees. 

This  form  of  the  value  table  is  satisfactory  for  eliciting 
the  assessments,  but  it  does  not  directly  provide  the  infor- 
mation needed  to  calculate  the  thresholds.  Table  1-3  shows 
a convenient  format  for  calculating  the  decision  thresholds. 
This  format  is  analogous  to  that  used  in  the  cutting-plane 
method  and  illustrated  in  Figures  0-21  and  0-22.  For  the 
weather  thresholds  shown  in  the  top  of  Table  3-3,  first  so 
importance-weighted  table  of  values  is  obtained  by  neltiplyln" 
each  entry  of  the  assessed  value  table  for  weather  (from 
Table  3-2)  by  its  importance  weight,  .33.  Nest,  the  voles 
of  each  action  must  be  adjusted  to  account  for  the  proba- 
bilities of  enemy  and  own-force  readiness.  This  adjustment 
is  calculated  by  weighting  the  assessments  in  the  enemy  sod 
own-force  readiness  value  tables  by  both  inportence  end 
probability  and  summing  across  rows.  For  example,  the 
adjustment,  due  to  enemy  and  own- force  readiness,  for  the 
action  "delay  1 day”  is  calculated  as  follows t 
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This  adjustment,  -11,  is  added  to  each  entry  in  the  "day 
delay"  row  of  the  weather  value  table,  and  the  other  adjust- 
ments are  calculated  in  a similar  manner  to  obtain  the  table 
from  which  the  weather  probability  thresholds  can  be  cal- 
culated, shown  on  the  right  of  Table  3-3. 

The  value  tables  for  thresholds  on  enemy  readi- 
ness and  own-force  readiness  are  calculated  in  a manner 
slightly  different  from  that  used  for  the  thresholds  on 
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EXPECTED 


Table  3-3:  STRIKE  TIMING  DECISION  - VALUE  TABLES 

FOR  THRESHOLD  CALCULATIONS 


weather.  The  difference  is  that  the  initial  value  table  for 
enemy  readiness  must  consider  the  probability  distribution 
over  states  of  own-force  readiness  and  vice  versa.  For 
example,  the  value  for  the  act ion/ state  combination  "No 
Delay /Enemy  Completely  Ready"  on  the  left  of  Table  3-3 


has  a value  of  -7  calculated 

as  follows 

using  data  from 

Figures  3- 

9 and  Table 

3-2: 
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The  adjustment  for  each  is  now  based  upon  the  weather  prob- 
ability, but  it  is  calculated  in  the  same  manner  as  the 
adjustment  to  the  weather  value  table.  For  example,  the 
adjustment,  due  to  weather,  for  the  action  "delay  1 day"  is 
calculated  as  follows: 


STATE  OF 
WEATHER 

ASSESSED 
VALUE  x 

IMPORTANCE 

X PROBABILITY  * 

PRODUCT 

Good 

-80 

.33 

.5 

-13 

Marginal 

-70 

.33 

.4 

-9 

Bad 

0 

.33 

.1 

TOTAL 

0 

-22 

The  value  tables  that  contain  the  necessary  information  for 
calculating  thresholds  are  shown  on  the  right  of  Table  3-3. 

The  effect  of  changing  the  probabilities  of  the 
states  of  weather  is  shown  in  Figure  3-10.  Increasing  the 
probability  of  marginal  weather  enough  for  the  "delay  1 
hour"  decision  to  be  preferred  changes  the  position  of  the 
probability  bug  in  the  weather  triangle  and  the  positions  of 
the  thresholds  in  the  enemy  and  own-force  readiness  tri- 
angles. This  change  in  weather  causes  the  entire  enemy 
readiness  triangle  to  change  from  favoring  "no  delay"  to 
favoring  "delay  1 hour."  In  the  own-force  readiness  tri- 
angle, the  action  space  for  "delay  1 hour"  was  greatly 
increased  while  the  action  space  for  "delay  1 day"  was 
decreased;  and  the  action  sptce  for  "no  delay"  was  elimi- 
nated (the  previous  thresholds  are  shown  as  dotted  lines  in 
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Figure  3-10) . Therefore,  at  this  new  assessment  of  weather, 
no  change  in  enemy  or  own-force  readiness  could  cause  the 
"no  delay”  action  to  be  favored,  and  only  a large  change  in 
the  assessed  state  of  own-force  readiness  could  cause  the 
"delay  1 hour"  action  to  be  favored. 

The  value  tables  that  were  used  to  calculate 
these  new  thresholds  are  shown  in  Table  3-4.  Since  the 
probabilities  of  enemy  and  own-force  readiness  remained 
constant,  the  only  changes  in  the  threshold  tables  are  in 
the  adjustments  due  to  weather,  which  are  added  to  the  rows 
of  the  enemy-readiness  value  table  and  the  own-force-readiness 
value  table. 

This  formulation  and  display  of  the  strike- 
timing decision  appears  to  be  a satisfactory  way  to  model 
the  decision  problem.  However,  our  investigation  also 
uncovered  an  alternative  formulation,  one  that  combines  the 
features  of  more  than  one  conditioning  tier  (described 
above)  and  more  than  three  states  within  a tier  (described 
in  section  3.1).  This  formulation  appears  to  provide  a more 
direct  elicitation  of  value,  and,  for  this  reason,  it  may  be 
a preferable  formulation  of  the  problem. 

Figure  3-11  shows  the  decision  tree  structure 
for  this  alternative  formulation  of  the  problem.  The  actions 
are  the  same  and  the  state  of  weather  are  the  same.  How- 
ever, the  states  of  enemy  and  own  force  readiness  are  different. 
Now,  the  net  readiness  of  the  enemy  (Red)  and  own  (Blue) 
forces  is  modelled.  This  net  readiness  is  characterized  in 
terms  of  five  states: 

1.  Blue  much  greater  than  Red, 

2.  Blue  greater  than  Red, 

3.  Blue  equal  to  Red, 

4.  Blue  less  than  Red, 

5.  Blue  much  less  than  Red. 

Figure  3-12  presents  the  graphic  display  of  the 
model  using  both  the  multiple-triangle  method  described 
above,  and  the  cutting-plane  method  (described  in  section 
3.1.4)  for  the  "force  readiness"  event  tier.  The  top  set  of 
triangles  shows  the  model  for  one  assessment  of  probabilities; 
the  middle  set  of  triangles  shows  the  changes  caused  by 
changing  the  weather  probabilities;  and  the  bottom  set  of 
triangles  shows  the  changes  caused  by  changing  the  force 
readiness  probabilities.  Table  3-5  shows  the  assessed  value 
tables  that  were  used  to  derive  the  thresholds.* 


9 

The  numbers  in  this  illustration  are  unrelated  to  those 
in  the  previous  illustration. 
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Figure  3-11 

ALTERNATIVE  STRIKE  TIMING  MODEL 
TREE  STRUCTURE 


The  investigation  into  tha  generalized  use  of 
the  Aid  resulted  in  the  identification  of  the  primary  cri- 
teria that  should  be  met  to  use  the  Execution  Aid.  First# 
states  should  be  independent  of  actions.  Second#  each 
action's  value  should  be  a function  of  the  states.  These 
conditions  appear  to  be  met  by  several  tactical  decision 
situations#  and  an  examination  of  the  ONRODA  scenarios 
revealed  that  the  strike  timing  decision  situation  met  these 
criteria.  An  adaptation  of  the  Execution  Aid  to  this  decision 
situation  required  a method  to  model  graphically  several 
tiers  of  conditioning  events.  This  requirement  was  met  by 
using  several  triangles#  and  initially  one  triangle  was  used 
for  each  event  tier.  A further  investigation  revealed  that 
the  problem  could  be  reformulated  in  a way  that  provided  an 
easier  value  elicitation  by  using  the  cutting-plane  method 
to  treat  more  than  three  states  at  one  of  the  conditioning 
tiers.  Further  investigation  of  this  situation  is  required 
to  determine  which  formulation  of  the  problem  is  the  more 
desirable.  However#  we  can  conclude  at  this  time  that  one 
of  these  formulations  will  lead  to  wider  applicability  of 
the  Execution  Aid  to  situations  other  than  the  ASM  threat  as 
a tactical  contingency. 

3.4.2  Tactical  grid  for  resource  allocation  - The 
investigation  described  in  Section  3.4.1  was  concerned  with 
identifying  a more  generalized  use  of  the  properties  of  the 
prototype  Execution  Aid.  By  contrast#  this  Section  reports 
the  results  of  an  investigation  of  a more  generalized  use  of 
the  underlying  principles  of  probability  and  utility  theory 
in  aids  for  task  force  commanders.  Accordingly#  we  did  not 
restrict  our  investigation  to  situations  where  probability 
thresholds  were  appropriate.  In  this  investigation#  we 
developed  a prototype  software  program  of  a new  aid  that 
could  assist  a task  force  commander  to  analyze  a problem 
such  as  allocating  fighter  sorties  to  targets.  This  section 
describes  this  aid  in  the  context  of  a decision  action  in 
assigning  fighter  sorties  to  targets  in  the  "Amphibious 
Warfare  Scenario"  (Rowney#  [1975]). 

In  this  example#  the  Commander  of  the  Supporting 
Naval  Forces  (CSNF)  in  the  amphibious  scenario  must  decide 
how  to  allocate  his  sixty-four  fighter/attack  aircraft  among 
the  following  nine  targets: 

1.  Greyport  Airfields 

2 . Greyport  Seaport 

3.  Close  air  support  (CAS)  at  the  Blue  FEBA  (FEBA  1) 

4.  CAS  at  the  Grey  FEBA  (FEBA  2) 

5 . Armed  recce 

6.  Fighter  sweeps 

7.  Orange  aircraft  re-supply 

8.  The  highway  and  train  bridge 

9.  Task  force  defense — CAP/SUCAP 


The  Commander  must  con a id a r tha  multiple  mission  requirements: 
to  support  the  landing  forces,  to  provide  task  force  defense, 
and  to  limit  damage  in  Grey.  This  decision  must  be  made 
during  each  of  the  major  phases  of  the  amphibious  operation, 
namely,  pre-assault,  assault,  and  post  assault.  Figure  3-13 
shows  a map  of  the  area,  with  several  of  the  targets 'Identified. 

The  principles  of  probability  and  utility  theory 
can  be  used  in  this  situation  to  determine  the  expected 
value  of  assigning  attack  sorties  to  the  targets  thereby 
contributing  to  the  preparation  of  a target  assignment  list. 

This  is  done  according  to  the  procedure  described  in  Table  3-6. 
The  expected  net  value  of  assigning  a single  sortie  to  a 
target  is  calculated  by  subtracting  the  expected  loss  from 
assigning  the  sortie  to  the  target  from  the  expected  value 
of  the  destruction  done  by  the  sortie  against  the  target. 

The  expected  loss  is  calculated  by  multiplying  the  value  of 
the  sortie  times  the  probability  that  it  will  be  lost.  The 
expected  value  of  the  destruction  is  calculated  by  multiplying 
the  value  of  the  target,  which  is  a weighted  value  that 
considers  several  objectives,  by  the  probability  that  a 
sortie  will  destroy  the  target.  The  net  value  of  assigning 
a second  sortie  to  the  target  is  calculated  by  a similar 
method  except  that  the  value  of  the  target  is  reduced  by  the 
expected  destruction  from  the  first  sortie.  The  value  of 
assigning  a third  sortie  is  calculated  considering  the 
expected  damage  done  by  the  first  two  sorties,  and  so 
forth.  For  example,  the  incremental  value  of  assigning  each 
of  1,  2,  and  3 sorties  to  a target  may  be  calculated  as 
shown  below: 
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Table  3-7  shows  the  inputs  used  in  the  target- 
selection  problem  in  the  "Amphibious  Warfare  Scenario.”  The 
top  table  shows  the  relative  values  assessed  for  each  target 
along  each  objective.  The  second  table  shows  the  importance 
weights  for  each  objective.  The  third  table  shows  the 
probability  that  a single  sortie  (a  sortie  is  defined  as  two 
attack  aircraft  in  this  problem)  will  destroy  the  target. 
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Figure  3-13 

TARGETS  IN  THE  AMPHIBIOUS  WARFARE  SCENARIO 


Table  3^6:  EXPECTED  NET  TARGET  VALUE  CALCULATION 
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Table  3-7:  TARGET  SELECTION  PROBLEM  - INPUTS 
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The  fourth  table  shows  ths  probability  that  a sortla  would 
be  lost  if  assigned  to  each  target.  The  fifth  (bottom) 
table  is  an  assessment  of  the  value  of  a sortie,  expressed 
in  the  same  units  of  value  as  the  top  table.  These  inputs 
will  remain  constant  during  all  three  phases  of  the  opera- 
tion: pre-assault,  assault,  and  post-assault.  The  final 

required  input,  the  weight  of  effort  devoted  to  each  objec- 
tive, however,  will  be  different  in  each  phase  of  the  opera- 
tion. 


Table  3-8  shows  the  weight  of  effort  input  for 
the  pre-assault  phase  and  the  output  calculated  for  this 
phase.  The  major  effort  is  devoted  to  carrying  out  the 
assigned  pre-assault  mission,  and  minor  effort  is  devoted  to 
task  force  defense  and  damage- limiting  in  Grey.  The  second 
table  displays  the  incremental  value  of  assigning  sorties  to 
the  targets  (calculated  as  explained  above) . Based  on  this 
information,  the  CSNF  can  develop  his  target  list  and  sortie 
assignments  to  maximize  the  net  expected  value  by  assigning 
sorties  to  targets  with  the  largest  positive  values. One 
optimal  allocation  of  the  32  sorties  is  shown  by  the  circled 
entries  in  the  target/sortie  value  table.  This  optimum 
allocation,  which  assigns  eight  sorties  to  the  airport, 
seven  to  the  seaport,  and  so  forth,  is  obtained  by  assigning 
the  first  31  sorties  so  as  to  obtain  incremental  net  value 
gains  of  three  or  more  and  by  assigning  the  32nd  sortie  to  a 
target  that  has  an  incremental  gain  of  two.  The  third  table 
shows  the  percent  of  target  damage  that  can  be  expected  for 
any  assignment  of  sorties.  For  example,  for  the  optimal 
assignment  described  above,  the  eight  sorties  assigned  to 
the  airport  are  expected  to  cause  an  85%  level  of  damage 
against  this  target,  the  seven  sorties  assigned  to  the  sea- 
port are  expected  to  cause  a 79%  level  of  damage  against 
this  target,  and  so  forth.  The  bottom  table  shows  the 
damage  expected  to  be  sustained  by  the  sorties  with  the 
target  assignments.  For  example,  with  the  optimal  alloca- 
tion from  above,  there  is  a 64%  chance  of  losing  a sortie 
over  the  airport,  a 7%  chance  of  losing  a sortie  over  the 
seaport,  and  so  forth.  The  CSNF  may  wish  to  consider  these 
items  as  well  in  making  the  sortie  allocation. 


10 Integer  programming  routines,  such  as  those  described  in 
Chapter  11  of  Wagner  (1970) , could  be  used  to  optimize  the 
solution,  but  such  a feature  is  not  included  in  the  present 
version  of  this  Aid.  Note  also  the  current  SRI  (Rowney) 
investigation  to  develop  an  air  warfare  engagement  outcome 
model. 


TARGET 


PERCENT  SORTIE  DAMAGE 


OR TIES 


Table  3-8:  TARGET  SELECTION  PROBLEM  - PRE-ASSAULT  PHASE 


Table  3-9  shows  how  ths  target/ sortis  value 
changes  as  the  weight  of  effort  changes  in  the  assault  and 
post-assault  phases  of  the  operation.  Based  on  these  re- 
vised values,  new  optimal  allocations,  such  as  those  circled 
in  the  tables,  are  obtained.  For  instance,  the  seaport  is  a 
high-value  target  for  the  pre-assault  and  assault  phases, 
but  not  in  the  post-assault  phase.  On  the  other  hand,  FEBA 
2 is  a low- value  target  for  the  pre-assault  phase,  but  it  is 
a high-value  target  for  both  the  assault  and  post-assault 
phases.  The  percent  target  damage  and  sortie  damage,  how- 
ever, do  not  depend  upon  the  weight  of  effort  and,  therefore, 
are  the  same  in  all  phases  of  the  mission  (these  quantities 
are  shown  in  the  bottom  two  tables  of  Table  3-8). 

This  example  illustrates  that  the  principles  of 
utility  and  probability  theory  can  be  combined  with  an  en- 
gagement outcome  model  into  an  aid  to  assist  the  task  force 
commander  in  real-time  decision  situations  (in  this  case,  to 
prepare  a target  list  for  air  strike  implementation) . The 
on-the-spot  nature  of  this  aid  is  emphasized  because  we  feel 
that  it  can  be  used  in  a short- lead- time  situations.  In 
fact,  the  example  displayed  above  required  only  a few  hours 
to  build  the  model  from  scratch  and  enter  all  of  the  data 
into  an  IBM  5100  micro-computer.  Changes  in  the  weights  to 
change  from  the  pre-assault  to  assault  to  post-assault 
phases  were  accommodated  instantaneously. 

3.4.3  Status  and  conclusions  - We  have  identified  an 
interactive  graphic  software  specification  for  the  multiple- 
triangle  display  used  in  the  strike-time  decision  (Section 
3.4.1),  but  no  computer  software  has  been  developed.  In 
addition,  we  have  identified  the  specification  for  a resource 
allocation  aid,  described  in  Section  3.4.2,  and  have  developed 
a prototype  software  program  of  this  aid  for  the  IBM  5100 
portable  microcomputer . This  software  prototype  was  demon- 
strated at  the  Operational  Decision  Aids  contractors'  meeting 
held  in  May,  1976. 

The  results  of  the  investigation  into  the 
generalised  use  of  decision  analysis  aids  for  the  task  force 
coosaander 1 s execution  phase  decisions  indicate  that  tactical 
aids  can  be  developed  for  other  than  ASM  threat  situations. 
These  aids,  however,  will  take  a form  slightly  different 
frosi  the  Execution  Aid  that  was  developed  within  the  context 
of  the  ASM  threat  situation. 

At  this  point  in  our  investigation,  decision 
analysis  appears  most  applicable  in  the  area  of  tactical 
response  to  contingency  situations.  Our  investigation 
indicates  that  probability  thresholds  have  general  applica- 
bility to  situations  in  which  states  are  probabilistically 
independent  of  actions  and  the  value  of  an  action  can  be 
assessed  directly  as  a function  of  the  states.  Several  task 
force  tactical  situations  appear  to  meet  the  conditions,  where 
major  actions  may  be  disrupted  by  uncertain  events.  It 
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DISTRIBUTION  OF  EFFORT 


PRE-ASLT  0 

ASSAULT  75 

POST  ASSAULT  0 ASSAULT 

TF  DEF  15 

DAM  LIM  10 


DISTRIBUTION  OF  EFFORT 

PRE-ASLT  0 

ASSAULT  0 

POST  ASSAULT  60  POST-ASSAULT 

TF  DEF  30 

DAM  LIM  10 


Table  3-9:  TARGET  SELECTION  PROBLEM  - ASSAULT  AND 

POST-ASSAULT  PHASES 
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appears,  though,  that  such  situations  require  a slightly 
different  form  of  the  prototype  Execution  Aid,  one  that  can 
accommodate  several  tiers  of  conditioning  events.  It  also 
appears  that  such  situations  are  characterized  by  states 
whose  probabilities  can  be  assessed  directly,  without  a 
Bayesian  hierarchical  model  for  indicators.  While  further 
investigation  is  required  before  the  exact  range  of  appli- 
cability of  this  additional  form  of  the  Aid  can  be  deter- 
mined, we  can  conclude,  at  this  time,  that  there  is  a wider 
applicability  of  the  prototypical  aid  to  situations  other 
than  defense  against  ASM  threats. 

Decision  analysis  principles  also  appear  to  be 
applicable  to  certain  other  task  force  decision  problems. 

In  particular,  our  investigation  indicates  that  decision 
analysis  principles  of  utility  and  probability  theory  can  be 
formulated  into  an  aid  that  can  be  used  on-the-spot  to  allo- 
cate resources,  for  example,  to  assign  attack  sorties  to 
targets.  This  aid,  however,  varies  significantly  from  the 
prototype  Execution  Aid.  In  particular,  the  allocation  aid 
does  not  provide  guidance  in  terms  of  probability  thresholds 
for  actions,  but  rather  provides  guidance  on  the  expected 
net  value  for  different  allocations  of  a resource.  We  feel 
that  this  aid  may  be  applicable  to  a lesser,  although  important, 
class  of  task  force  decision  situations  than  the  Execution 
Aid,  but  that  it  does  demonstrate  an  expanded  application  of 
decision  analysis  principles  to  a tactical  commander’s 
execution-phase  decisions. 

Further  investigation  is  needed  to  determine 
definitively  the  range  of  applicability  of  these  decision 
aids  to  tactical  decision  problems,  and  such  an  effort  is 
proposed  as  a future  task.  However,  we  have  informally 
examined  the  task  force  commander's  execution-phase  decision 
situations  presented  in  three  documents,  Payne  et  al.  (1974), 
Payne  and  Rowney  (1975) , and  Rowney  (1975) , and “Have  tenta- 
tively concluded  that  the  decision  aids  we  have  identified 
are  applicable  in  some  form  to  at  least  fifteen  of  the 
situations  presented.  This  inquiry  is  described  in  Appendix 
A of  this  report  and  will  be  expanded  under  a proposed  con- 
tract phase. 
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4.0  SUMMARY  OF  ACCOMPLISHMENTS , CONCLUSIONS 
AND  RECOMMENDATIONS 


4.1  Summary  of  Accomplishment* 

1.  An  evaluation  questionnnaire,  baaed  on  a list  of 
desired  aid  characteristics/  was  developed  for  use 
in  the  pilot  test.  This  questionnaire  proved  to 
be  a convenient  and  efficient  instrument  for 
recording  the  subjects'  judgments  about  the  proto- 
type Execution  Aid, 

2.  The  range  of  "enemy"  ASM  threats  was  modeled  in  a 
form  that  could  be  incorporated  into  the  Execution 
Aid.  These  threats  were  both  the  small-scale 
air/submarine  threat  and  the  large-scale  ASM 
attack  system  (containing  air,  surface,  and  sub- 
surface attack  elements) . The  modeling  form 
consisted  of  a hierarchical  tiering  of  threat 
activity  and  a list  of  indicators  of  "enemy" 
intent,  with  the  air/ submarine  model  only  imple- 
mented in  computer-graphic  software. 

3.  A pilot- test  procedure,  consisting  of  a sequence 
of  off-line  (view-graph)  and  on-line  (interactive 
computer-graphic)  interactions  was  developed. 

This  procedure  proved  to  be  an  effective  way  to 
both  enhance  comprehension  of  the  aid  and  facili- 
tate its  use  by  the  subjects  tested. 

4.  Pilot  testing  was  conducted,  and  the  conceptual 
acceptance  of  the  Execution  Aid  was  determined. 

In  addition,  the  pilot  testing  identified  areas 
for  software  refinement. 

5.  Based  upon  the  pilot  test,  modifications  were  made 
in  the  interactive  computer-graphic  Aid  software. 

6.  A user's  manual  is  being  prepared  for  the  computerized 
Execution  Aid. 

7.  An  investigation  into  methods  of  modifying  the 
Aid's  display  to  accommodate  more  than  three  state 
hypotheses  was  completed.  The  software  modifica- 
tions necessary  to  utilize  this  feature  were 
identified,  but  they  have  not  yet  been  implemented. 

8.  Initial  investigations  have  been  made  into  methods 
of  incorporating  explicit  considerations  of  time 
and  risk  into  the  Aid  with  inconclusive  results  at 
this  writing. 
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9.  A method  to  display  graphically  more  than  one  tier 
of  conditioning  events  has  been  identified. 

10.  A resource  allocation  aid  that  utilizes  decision 
analytic  principles  has  been  developed.  Prototype 
computer  software  has  been  developed  for  this  aid 
on  the  portable  IBM  5100  microcomputer. 

11.  The  usefulness  of  a decision  aid  in  situations 
other  than  reaction  to  the  ASM  threat  has  been 
established. 

12.  A tentative  matching  of  prototype  decision  aids  to 
task  force  decision  situations  has  been  performed. 

4.2  Conclusions 


i 


Our  research,  development/  and  testing  to  date  has 
resulted  in  several  conclusions  about  both  the  acceptability 
and  the  range  of  application  of  decision  analysis  to  a Navy 
task  force  commander's  mission  execution  decisions. 

The  results  of  the  pilot  testing  effort  indicate  that 
the  concepts  involved  in  our  decision  analytic  Execution  Aid 
are  acceptable  to  Navy  task  force  commanders.  These  results 
lead  us  to  conclude  that  the  prototype  Aid  is  now  ready  to 
undergo  more  thorough  evaluative  testing  in  a controlled 
experiment. 

The  investigations  of  the  ASM  threat  indicate  that  this 
threat  can  be  adequately  modeled  in  a Bayesian  hierarchical 
analysis  and  the  Aid  can  usefully  monitor  the  threat  as  it 
develops. 

The  investigations  into  the  generalized  use  of  the  Aid 
indicate  that  it  is  applicable  to  a broader  class  of  decision 
situations  than  those  involving  ASM  threat  alone.  In 
particular,  the  Aid  appears  to  be  applicable  to  a wide  range 
of  contingencies,  that  is,  uncertain  events  that  might 
warrant  a change  in  operational  plan.  The  probability 
threshold  concept  appears  to  be  especially  applicable  to 
contingency  situations  in  which  state  probabilities  are 
independent  of  the  possible  actions  and  value  is  a function 
of  the  action/state  combinations. 

Finally,  it  appears  that  it  is  possible  to  extend  the 
graphic  display  of  the  probability  space  to  consider  more 
than  one  tier  of  conditioning  events  and  more  than  three 
state  hypotheses  within  a tier. 
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4.3  Recommendations 


In  our  view,  th«  Operational  Decision  Aids  project  has 
reached  a point  in  time  when  the  presently  identified  decision 
analytic  tools  warrant  some  form  of  systematic  evaluation  of 
their  potential  usefulness  beyond  their  conceptual  acceptance. 
At  the  same  time,  it  appears  prudent  to  continue  a general 
investigation  of  the  decision  analysis  technology  for  the 
purpose  of  identifying  an  increasing  number  of  promising 
aids  embodying  proven  properties  of  the  methodology.  Based 
upon  our  current  research,  we  have  identified  the  following 
areas  as  promising  for  the  future  testing,  research,  and 
development  of  decision  analytic  aids  for  Navy  task  force 
operations. 


4.3.1  Decision  aid  test  and  evaluation  - In  order  to 
establish  the  usefulness  of  the  Execution  Aid,  we  feel  that 
it  is  important  to  perform  a small-scale  controlled  simula- 
tion experiment.  Such  an  experiment  can  provide  many  of  the 
useful  insights  of  a large-scale  simulation  at  a much  lower 
cost.  In  addition,  a Fleet  Operational  Investigation  of  the 
emerging  decision  aid  prototypes  is  recommended  as  a com- 
plementary and  synergistic  ODA  project  undertaking.  Based 
on  several  fleet  inquiries  as  to  the  purpose  and  objectives 
of  the  ODA  project,  it  is  deemed  appropriate  to  establish  a 
limited  or  controlled  fleet  evaluation  of  those  prototype 
operational  decision  aids  that  have  achieved  at  least  con- 
ceptual acceptance.  The  fleet  evaluation  would  be  prepared 
bv  ONR  under  the  sponsorship  of  OP-094/943  (Command  and  Control) 
in  conjunction  with  OP-098/983  (Test  and  Evaluation) , and  the  pro 
totype  decision  aids  would  be  demonstrated  for  operational  feasi- 
bility short  of  a formal  technical  and  operational  evaluation. 

The  major  purposes  of  the  investigation  are  to  provide 
operational  identification  of  the  project,  to  provide  a 
realistic  environment  for  early  operational  assessment 
(enabling  a better-focused  research  effort) , and  to  test  the 
degree  of  potential  application  of  the  aid  as  tactical 
decision  tool. 

We  also  feel  that  the  aid  should  be  given  exposure, 
for  the  purpose  of  evaluation,  in  the  Naval  training  environ- 
ment. In  recent  years,  the  Navy  has  placed  increasing  emphasis 
on  the  use  of  its  war-gaming  and  tactical  simulation  facilities 
for  the  purpose  of  training  naval  officers  to  better  comprehend 
and  cope  with  the  accelerating  sophistication  in  tactical 
weaponry  and  with  the  corresponding  threat  to  naval  forces. 

The  Center  for  War  Gaming  at  the  Naval  War  College  has 
steadily  improved  its  gaming  environment  to  permit  the 
complex  play  of  multi-dimensional  tactical  problems  in 
crisis  or  combat  situations  that  are  realistically  portrayed 
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to  tactical  decision  makers.  The  decision  environment 
generated  by  the  war  gaming  facility  offers  a rich  inte- 
gration of  all  significant  factors  of  own  force,  enemy 
force,  and  environmental  conditions  that  are  dynamic  and 
interactive  and  are  encountered  by  tactical  decision  makers 
in  the  course  of  task  force  operations.  The  evolving  system 
is  highly  flexible,  and  by  including  human  decision-making 
in  the  war  game,  it  is  possible  to  create  tactical  decision- 
making situations  most  likely  to  be  encountered  in  real- 
world  operations.  Without  question,  war  gaming  at  this 
level  of  sophistication  and  realism  offers  an  exceptional 
opportunity  for  testing  and  evaluating  new  system  concepts 
designed  to  support  the  command  and  control  of  naval  forces 
engaged  in  crisis  or  combat  situations. 

In  a similar  manner,  the  tactical  simulation 
facilities  of  the  Fleet  Training  Centers  have  been  reoriented 
in  recognition  of  the  fast  moving  tactical  situations  inherent 
in  the  single-dimension  warfare  areas  such  as  AAW,  ASW,  and 
EW  where  multi-threat  conditions  exist.  Here,  as  in  the  war 
gaming  environment  at  the  Naval  War  College,  realistic 
tactical  situations  are  developed  for  real-time  decision 
making  in  which  human  beings  are  interacting  with  command 
support  systems  to  optimize  their  decision  choices  for 
tactical  action.  In  both  settings,  we  perceive  a unique 
opportunity  to  observe  and  to  investigate  the  realistic 
environment  of  tactical  decision  making  and  thereby  determine 
user  requirements.  Additionally,  and  in  the  case  of  the  War 
Gaming  Center  in  particular,  there  is  a clear  opportunity  to 
integrate  promising  decision  aids  into  the  evolving  war 
gaming  system  and  to  evaluate  their  contribution  to  the 
tactical  decision  environment  being  generated.  As  an  added 
bonus,  both  training  situations  enjoy  high-caliber  student 
participation  at  an  observed  competitive  level,  and  this  may 
well  provide  insight  as  to  where  decision-aiding  tools  of  a 
computer-graphic  nature  will  have  maximum  utility. 

4.3.2  Research  and  development  - In  the  course  of  our 
current  investigations,  we  have  identified  numerous  properties 
of  decision  analysis  methodology  that  appear  to  be  applicable 
to  the  task  force  commander's  decision  environment.  In 
developing  the  decision  aids,  we  initially  focused  on 
combining  these  elements  into  a single  aid.  However,  we  are 
now  of  the  opinion  that  an  improved  capability  can  be 
achieved  if  future  aids  are  developed  in  a modular  fashion 
by  using  elements  of  decision  analysis  in  various  aid  combi- 
nations appropriate  to  the  decision  problem.  Consequently, 
we  feel  that  the  following  software  modules  warrant  immediate 
development: 
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1.  A module  to  perform  and  display  a Bayesian  hier- 
archical analysis. 

2.  A module  to  implement  the  cutting-plane  method  of 
modeling  more  than  three  state  hypotheses. 

3.  A module  for  modeling  several  tiers  of  conditioning 
events. 

In  addition  to  the  development  activity  described 
abovef  we  recommend  further  investigation  into  the  generalized 
use  of  the  prototype  aids  and  the  related  area  of  matching 
modular  aids  to  task  force  decision  situations.  It  appears 
at  this  time  that  the  various  properties  of  decision  analysis 
used  in  an  independent  or  combined  form  (modularity) , offer 
unique  potential  for  the  development  of  powerful  decision- 
aiding  tools.  Whether  a universal  decision  aid,  that  is, 
the  more  general  use  of  a single  aid,  or  a modular  tool 
tailored  through  simple  combination  rules  is  the  more  appro- 
priate in  application  to  tactical  decision  problems  requires 
further  investigation  and  analysis.  It  is  hoped  that  an 
examination  of  a variety  of  tactical  situations  (for  instance: 
threat,  strike,  transit,  force  positioning,  target  selection, 
and  so  forth)  will  establish  a basis  for  developing  matching 
principles  so  that  tactical  decision  tools  can  be  quickly 
assembled  for  tactical  decision  making. 


APPENDIX  A 


MATCHING  AIDS  TO  SITUATIONS 


A.l  Introduction 

At  the  direction  of  the  Scientific  Officer,  we  under- 
took the  taek  of  investigating  three  of  the  ODA  project 
reports,  "ONRODA  Warfare  Scenario"  (Payne  and  Rowney  [ 1975 J ) , 
"Amphibious  Warfare  Scenario"  (Rowney  [1975]),  and  "The 
Naval  Task  Force  Decision  Environment"  (Payne  et  al.  [1974]), 
to  identify  task  force  commander  decision  situations  to 
which  the  prototype  decision  aids  described  in  this  report 
would  apply.  The  investigation  has  been  informal  and  re- 
quires further  work  before  definitive  conclusions  can  be 
reached.  However,  we  can  tentatively  conclude  that  the 
decision  aids  described  in  this  report  are  applicable,  to 
some  extent,  to  at  least  fifteen  of  the  decision  situations 
that  were  investigated.  This  appendix  serves  as  the  starting 
point  for  a more  extensive  investigation  to  be  conducted 
under  a proposed  contract  phase. 


A. 2 Scope  of  the  Matching  Investigation 

We  have  limited  the  scope  of  this  matching  investi- 
gation to  tactical  decision  situations  in  which  one  of  the 
three  aids  described  in  this  report,  the  original  Execution 
Aid,  the  Contingency  Aid  (see  Section  3.4.1  above) , and  the 
Tactical  Grid  (see  Section  3.4.2  above),  or  some  variant  of 
these  aids  could  be  applied.  While  other  decision  situa- 
tions may  be  important  to  the  task  force  commander,  we  feel 
that  decision  analysis  may  prove  more  beneficial  to  situa- 
tions in  which  some  form  of  the  Contingency  Aid  (either  the 
Execution  Aid  or  the  Contingency  Aid  of  Section  3.4.1) 
applies. 

In  particular,  we  feel  that  contingency  situations 
involving  naval  mission  execution  offer  a more  fruitful  area 
for  decision  analytic  investigation  than  mission  planning 
decisions.  Previous  investigations  revealed  that  it  was 
difficult  to  identify  initial  planning  decisions  that  were 
rich  enough  to  support  a decision  analysis  effort  beyond 
planning  for  allocation-type  problems.  In  contrast,  con- 
tingency situations  appear  to  be  characterized  by  value 
conflicts  and  by  a wide  range  of  action  possibilities  that 
make  them  fruitful  areas  for  application  of  decision  analysis 
in  tactical  environments.  Within  a class  of  contingency 
decisions,  it  is  possible  to  develop  decision  thresholds 
based  on  events,  such  as  "Do  not  shoot  until  you  are  fired 
upon"  or  "Avoid  confrontation  at  all  cost.”  In  some  situa- 
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tions  these  thresholds  may  provide  useful  guidance,  and  in 
those  situations  decision  analysis  is  not  very  applicable. 
However/  in  many  situations  these  types  of  thresholds  are 
too  rigid  to  be  useful  as  action  guidelines/  and  here  decision 
analysis  can  be  helpful  by  providing  probability  thresholds 
that  allow  the  combined  effect  of  many  combinations  of  data 
to  indicate  the  preferred  action.  Finally,  probability 
thresholds  for  actions  may  be  either  dependent  or  independent 
of  the  specific  mission  context  in  which  a choice  must  be 
made.  Computerized  decision  aids  appear  to  be  most  appli- 
cable for  developing  and  displaying  probability  that  is 
independent  of  the  mission  context  because,  in  such  cases, 
useful  computerized  models  can  be  developed  in  advance. 

Thus,  our  investigation  of  the  decision  situations 
described  in  the  three  referenced  documents  focused  on 
classifying  mission-execution-phase  decisions  according  to 
the  applicability  of  probability  thresholds  and  the  degree 
of  scenario  independence.  Secondarily,  the  situations  were 
classified  according  to  the  applicability  of  several  of  the 
specific  analytic  options  available  in  the  aids  under  develop- 
ment. 


A. 3 Matching  Analytic  Options  to  Situations 

The  first  step  in  matching  potential  decision  aids  to 
situations  is  to  match  the  analytic  options  that  are  offered 
in  the  aids  to  the  situations.  Table  A-l  displays  the 
results  of  our  investigation  of  nine  situations  in  the 
ONRODA  scenario,  eight  situations  in  the  Amphibious  scenario, 
and  six  mission-execution-phase  decisions  from  the  Task 
Force  Decision  Environment  report.  These  situations  were 
chosen  on  the  basis  of  a high  expectation  of  finding  a match 
between  the  situations  and  the  analytic  options.  A match 
between  an  analytic  option  and  a situation  is  indicated  by 
an  X in  the  row  of  the  analytic  option  and  the  column  of  the 
situation.  For  example,  the  first  decision  situation,  the 
choice  of  a course  of  action  in  the  ONRODA  scenario,  is  felt 
to  support  the  use  of  probability  thresholds  derived  from  an 
analysis  that  contains  several  tiers  of  conditioning  events 
with  more  than  three  states,  a direct  assessment  of  state 
probabilities,  and  multiple  conflicting  values.  This  chart 
served  as  the  basis  for  matching  the  aids  to  situations  as 
explained  below. 

A. 4 Matching  Decision  Aids  to  Situations 

An  examination  of  Table  A-l  reveals  that  several  combi- 
nations of  analytic  options  are  applicable  over  a wide  range 
of  decision  situations.  These  combinations,  which  correspond 
roughly  to  the  prototype  aids  that  we  have  developed,  can 
thus  be  used  to  describe  decision  aids  that  are  applicable 
to  a number  of  situations.  These  combinations  are  as  follows: 
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Table  A 1:  MATCHING  ANALYTIC  OPTIONS  TO  SITUATIONS 


1.  Aids  that  use  decision  thresholds 

a.  Contingency  decision  aid — Mode  1 (the  original 
Execution  Aid) 

b.  Contingency  Decision  Aid — Mode  2 (the  generalized 
Contingency  Aid,  Section  3.4.1) 

c.  Contingency  Decision  Aid-reduced  form 

2.  Aids  that  do  not  use  decision  thresholds 

a.  Tactical  Grid — resource  allocation  (from 
Section  3.4.2) 

b.  Other  (undefined)  aid 

Table  A-2  illustrates  how  these  aids  match  the  situations 
considered. 

As  discussed  in  Section  3.4.1,  aids  with  decision 
thresholds  are  applicable  to  decision  situations  that  can 
be  structured  so  that: 

1.  The  state  probabilities  are  independent  of  the 
actions;  and 

2.  The  value  of  the  actions  can  be  assessed  as  a 
function  of  the  states. 

The  mode  1 type  contingency  aid(s),  which  incorporated  features 
of  the  original  Execution  Aid,  is  most  applicable  in  situa- 
tions that  are: 

1.  Predictable  and  scenario-independent  enough  to 
model  in  advance. 

2.  Characterized  by  a predictable  set  of  indicators 
that  can  be  modeled  in  a Bayesian  hierarchical 
structure  and  used  for  probability  updating. 

3.  Characterized  by  a state  space  that  can  be  adequately 
modeled  in  a single  conditioning  tier  that  contains 
three  or  fewer  states. 

Situations  such  as  the  "enemy"  ASM  threat  in  a crisis  setting 
fall  into  the  category  where  the  mode  1 contingency  aid  is 
most  useful.  Although  there  may  be  only  a relatively  small 
number  of  these  situations,  we  feel  that  they  are  most 
important  and  are  undoubtedly  contingencies  that  a Navy 
task  force  commander  would  encounter. 
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Table  A 2:  MATCHING  DECISION  AIDS  TO  SITUATIONS 


The  mod*  2 type  contingency  decision  aid(s)  is  composed 
of  features  that  were  developed  in  the  generalisation  sections 
of  this  report,  especially  Sections  3.4.1  and  D.5.  Namely, 
this  type  aid  has  provisions  for  multiple  conditioning 
tiers,  more  than  three  states  within  a tier,  and  multiple 
dimensions  of  value,  but  does  not  have  provisions  for  Bayesian 
probability  updating  or  Bayesian  hierarchical  analysis. 

This  aid  is  most  useful^  in  situations  that  are: 

1.  Predictable  and  scenario- independent  enough  to 
model  in  advance. 

2.  Not  characterized  by  a predictable  set  of  indica- 
tors for  Bayesian  probability  updating  (that  is, 
characterized  by  probabilities  that  can  be  assessed 
directly) . 

3.  Characterized  by  a complex  state  space  that 
requires  several  tiers  of  conditioning  events  or 
more  than  three  states  within  a tier. 

We  expect  more  of  the  commander's  decision  situations  to 
fall  into  this  category;  however,  these  decisions  may  not  be 
as  critical  as  the  ones  for  which  the  mode  1 aid  is  appropriate. 

Situations  in  which  a reduced  form  of  the  aid  such  as 
the  multi-attribute  value  tables  might  be  useful  involve 
either  simple  or  uncritical  decisions  that  do  not  deserve  a 
large  analytical  effort. 

Situations  that  cannot  be  adequately  modeled  in  a form 
that  is  appropriate  for  development  of  probability  thresholds 
require  aids  different  from  the  contingency  type  aids  described 
above.  One  such  aid  is  the  Tactical  Grid,  which  is  described 
in  Section  3.4.2.  This  aid  requires  the  decision  situation 
to  fit  a special  form  such  as  allocating  attack  sorties  to 
targets  in  target  selection  decisions.  Situations  that  do 
not  meet  any  of  these  requirements  need  other  forms  of 
decision  aids,  ones  which  we  have  not  yet  identified  or 
defined. 
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APPENDIX  B 

TEST  PROCEDURE  ILLUSTRATIONS 


B.l  Scenario  Familiarization 

A 35  mm  slide  presentation,  supported  by  a typical 
staff  intelligence  (N2)  and  operations  (N3)  officer  situa- 
tional brief,  was  presented  to  the  test  subject  as  a "scene 
setter"  for  ONRODA  scenario  familiarization.  The  information 
was  presented  in  a manner  calculated  to  provide  the  general 
political-military  situation  prevailing  at  the  time  the  Blue 
commander  received  his  mission  directive  from  higher  authority. 
The  subject  was  permitted  time  to  consider  the  operational 
planning  phase  that  followed.  The  "scene-setter"  then 
proceeeded  into  the  mission  execution  phase  of  the  scenario 
and  brought  the  test  subject  to  the  point  of  tactical 
execution  of  his  mission  plan. 

At  this  point,  the  ASM  threat  was  introduced  as  a 
tactical  situation  of  serious  concern  to  the  commander, 
requiring  on  his  part  a number  of  difficult  decisions,  a 
need  to  consider  significant  changes  to  mission  plan  and 
subordinate  tasking,  as  well  as  a review  of  any  inhibiting 
aspects  of  the  rules  of  engagement  as  imposed  by  higher 
authority. 

B.2  Off- line/ On-line  Interactions 

As  described  in  Section  2.3.2,  the  test  procedure 
involved  a series  of  off-line  (vu-graph  presentations)  and 
on-line  (computer  graphic)  interactions  between  the  subject 
and  the  Aid.  The  charts  used  in  the  off-line  presentations 
successfully  communicated  the  Aid's  properties  and  features 
to  the  subject  and  served  to  transition  the  subject  to  the 
computer -graphic  presentations  manifest  in  the  prototype  Aid 
for  ease  of  evaluation. 

B.2.1  Off-line  presentations  - The  following  visual 
aids  were  used  off:- line  to  introduce  the  subjects  to  the  Red 
ASM  threat  situation  that  would  be  presented  in  the  most 
reduced  form  of  the  Aid:  Table  B-l  describes  the  operational 

concerns  within  the  threat;  Table  B-2  lists  the  ASM  threat 
indicators  that  would  appear  as  the  situation  developed;  and 
Table  B-3  presents  a checklist  of  some  of  the  actions  that 
the  subject  (serving  as  the  Blue  task  force  commander)  might 
wish  to  take  in  response  to  the  developing  threat. 

The  next  set  of  visual  aids  were  used  off-line 
to  describe  the  probability  plane  to  the  subjects  before 
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ROUTINE  - RED  INTENDS  TO  ENGAGE  IN  ONLY  HIS  ROUTINE 

SURVEILLANCE  ACTIVITIES. 


FEINT  _ RED  INTENDS  TO  BRING  HIS  AIR/SUBMARINE 

COMBINATION  TO  A FULL  STATE  OR  READINESS 
BUT  STOP  SHORT  OF  ATTACK. 


ATTACK  - RED  INTENDS  TO  ATTACK  THE  BLUE  TASK  FORCE 

USING  AN  AIR/SUBMARINE  COMBINATION. 


Table  B-1:  POSSIBLE  RED  OPERATIONAL  OPTIONS 


77 


Political 

National 

Deployment 

Surface 

Comma 

Submarine 

Search 

Targeting 

Track 

Beacon  Test 
Guidance  Radar  - 
Command  Comma 
Sounds 
Intercept 


Hostile 

Normal 

Ignoring 

General  Alert 
Theatre  Alert 
Selective  Alert 
No  Alert 

Tactical 

Strategic 

Normal 

Attack 

Withdraw 

Increase/High 

Normal/High 

Increase/Low 

Normal/Low 

Normal 

Multiple 

Yea 

NO 

MW/Int. 

MW/Steady 

None 

Routine 

Attack 

Yes 

NO 

Yes 

No 

Yes 

NO 

Yes 

NO 

Yes 

No 


Table  B-2: 


RED  ASM  THREAT  INDICATORS 


Table  B 3:  POSSIBLE  BLUE  RESPONSE  OPTIONS 
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this  feature  of  the  Aid  was  prasantad  in  on- 1 in  a interaction 
with  tha  graphic  terminal.  Figure  B-l  was  used  to  explain 
how  the  triangle  represents  probabilities#  in  effect#  how  to 
"read"  the  probability  triangle.  Figure  B-2  was  used  to 
display  the  beginning  probabilities  that  would  be  used  in 
the  computer  interaction. 


The  final  set  of  visual  aids  was  used  off-line 
to  introduce  the  test  subjects  to  the  action  threshold 
feature  of  the  Aid  before  on-line  interaction  with  the  full 
form  of  the  Aid.  Table  B-4  describes  the  criteria  that 
might  be  of  concern  to  the  commander  in  the  crisis  situation 
prevailing.  These  are  the  criteria  used  in  the  computer 
version  of  the  Aid. 

Table  B-5  was  used  to  describe  how  action 
thresholds  are  calculated  from  a value  matrix.  Figure  B- 3 
in  this  group  illustrates  how  a threshold  equation  is  dis- 
played in  the  probability  triangle. 

Finally#  Table  B-6  was  used  to  describe  the 
elicitation  process,  the  meaning  of  the  assessments  on  each 
value  criterion#  and  the  way  that  the  combined  value  matrix 
is  calculated.  The  values  in  Table  B-6  are  the  same  as 
those  initially  used  in  the  computer-graohic  interaction. 
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VALUE  CRITERIA 


Table  B-4:  COMMANDER'S  CONCERNS 
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C' 
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CM 


M 
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Criterion  Importance  Height 


APPENDIX  C 


QUESTIONNAIRE 


Timeliness  of  Decision 


How  might  the  aid  specification  be  modified  to  enable  a 
faster  and/or  more  complete  response  to  a developing 
threat? 

Do  you  feel  the  aid  would  serve  to  make  you  more  con- 
fident of  your  decision  choices  having  developed  tac- 
tical options  in  advance  of  a contingency? 

Are  you  concerned  that  tactical  situations  are  too 
dynamic  to  entrust  event  planning  to  a tool  that  is 
based  on  prior  planning  and  preprogramming? 


Quality  of  Decision 


Do  you  feel  this  aid  will  help  a CTF  focus  on  those 
decision  situations  that  are  of  primary  concern  to  him? 

Could  there  be  an  assured  continuity  of  expertise 
inherent  in  an  aid  of  this  type? 

Would  there  be  an  objection  to  the  "documentary" 
aspect  of  the  aid  insofar  as  a record  of  tactical 
action  is  concerned? 


Ease  of  Operation 


Do  you  feel  the  aid  contributes  to  the  essential  flow 
of  tactical  data,  or  is  it  in  some  way  an  inhibiting 
and  perhaps  unnecessary  tool? 

Perhaps  the  aid  would  better  serve  those  staff  officers 
responsible  for  the  functional  activities  of  mission  / 
accomplishment,  i.e.,  strike  planning,  AAW,  ASW,  etc. 

Do  you  feel  the  CTF  would  use  a decision  aid  of  this 
type  in  a tactical  situation? 
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1.  Of  the  three  displays,  is  there  any  difficulty  in 
comprehens ion? 

2.  Would  you  recommend  more  or  less  graphic  support? 


3.  Would  you  comment  on  the  need  for  an  executive  (CTF) 
aid  vis-a-vis  those  aids  the  supporting  staff  might 
use? 


B 


J. 


APPENDIX  D 


INVESTIGATIONS  FOR  TREATMENT  OF  MORE 
THAN  THREE  STATE  HYPOTHESES 
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APPENDIX  D 


INVESTIGATIONS  FOR  TREATMENT  OF  MORE  THAN 
THREE  STATE  HYPOTHESES 


D.l  Introduction 

As  stated  previously  in  Section  3.0/  the  geometry 
utilized  in  the  current  Execution  Aid  permits  only  three 
state  hypotheses  to  be  considered  in  our  decision  modeling. 

In  response  to  inquiries  regarding  situations  that  could 
generate  the  need  to  consider  more  than  three  hypotheses/ 
we  undertook  an  investigation  to  examine  alternative  methods. 
The  following  sections  describe  our  initial  investigations 
in  this  area. 

D. 2 Hierarchical  Arrangement  of  Models 

In  a hierarchical  arrangement  of  models/  the  problem  is 
addressed  in  stages/  each  of  which  has  its  own  separate 
model.  For  instance/  the  two  threat  situations  described  in 
Section  2.2  above,  that  is,  the  air/submarine  ASM  threat  and 
the  multiple-attack  element  ASM  threat,  might  be  addressed 
with  the  hierarchical  arrangement  of  models  shown  in  Figure 
D-l . At  the  top  level  of  the  hierarchy,  an  analysis  is  made 
of  whether  it  is  better  to  take  an  action  responsive  to  a 
single  threat,  a multiple  threat,  or  no  significant  threat 
(status  quo).  This  analysis  will  contain  values  and  indi- 
cators relevant  to  this  higher-order  decision.  A developing 
situation  will  cause  the  probability  point  (bug)  to  enter  a 
decision  area  such  as,  "respond  to  single  threat."  This 
event  will  then  trigger  the  recall  and  display  of  the  lower- 
level  or  subordinate  model  of  the  single  ASM  threat,  where 
the  subordinate  model  contains  values  and  indicators  relevant 
to  the  analysis  of  lower-order  decisions. 

This  hierarchical-arrangement-of-models  approach  is 
capable  of  handling  more  than  three  states  and  may  be  quite 
natural  in  many  situations,  but  its  structure  is  not  completely 
general.  In  particular,  it  may  prove  difficult  for  some 
decision  makers  to  model  their  problems  in  a manner  that 
fits  the  structural  requirements  of  the  hierarchical  arrange- 
ment and  yet  still  accurately  represent  the  actual  situation. 
This  approach  does  have  the  desirable  feature  that  each 
analysis  is  completely  described;  that  is,  nothing  is  omitted 
from  any  display,  and  there  is  no  theoretical  limit  to  the 
number  of  states  which  can  be  modeled. 


D.3  Dlstance-to-Threshold  Method 


An  alternative  way  to  model  more  than  three  states  is 
to  display  the  distances  from  the  probability  bug  to  the 
decision  thresholds.  A display  of  the  distances  to  the 
decision  thresholds  discards  the  triangular  display  entirely, 
and  it  does  not  restrict  the  analysis  to  any  particular 
form.  The  concept  and  derivation  of  the  distance-to-threshold 
display  are  illustrated  in  Figure  D-2.  The  top  figure  shows 
a probability  triangle  divided  into  five  action  regions, 

"R";  "S";  "H";  "P"?  and  "A".  At  the  probability  point 
indicated,  "P"  is  the  preferred  action  and  the  minimum 
distances  to  the  other  action  areas  are  indicated  as  the 
lengths  of  the  line  segments  Dr,  Ds,  D^,  and  Da.  The  bottom 
figure  shows  the  corresponding  distance-to-threshold  display. 
The  distance  to  the  preferred  action  is  zero,  and  the  dis- 
tances to  the  other  actions  are  those  calculated  as  shown  in 
the  top  figure.  Note  that  although  the  figure  illustrates 
the  concept  using  only  three  states,  the  display  of  the  dis- 
tances to  the  thresholds  requires  only  that  the  distances  to 
the  thresholds  can  be  calculated,  which  does  not  place 
restrictions  on  the  number  of  states.5 

Figure  D-3  illustrates  the  distance-to-threshold  display 
changes  as  the  probability  bug  crosses  into  the  "A"  action 
area. 

People  who  were  familiar  with  the  triangular  display 
form  participated  in  workshop  trials  using  the  distance-to- 
threshold  display.  These  trials  revealed  that  the  distance- 
to-threshold  display  lacked  the  user  appeal  of  the  triangular 
display  and,  more  importantly,  that  the  distance-to-threshold 
display  did  not  communicate  the  tactical  situation  as  well. 
Subjects  felt  that  the  display  did  not  communicate  sufficient 
information  about  the  situation.  In  particular,  they  indi- 
cated that  it  lacked  the  dynamic  and  the  geometric  quality 
of  the  triangular  form.  This  is  possibly  because  the  distance- 
to-threshold  display  does  not  provide  a graphic  history  of 
the  probabilities,  as  does  the  plot  of  the  probability  bug. 

On  the  whole,  this  alternative  appeared  to  be  undesirable. 

D. 4 The  Probability  Tetrahedron 

Both  the  projection  method  and  the  cutting-plane  method 
involve  the  concept  of  a probability  tetrahedron.  The 
probability  tetrahedron  is  a generalization  of  the  proba- 


Fishburn,  Murphy,  and  Isaacs  (1968)  presents  an  algorithm 
for  calculating  distances  to  thresholds. 


bility  triangla,  axtandad  to  thraa  diaansions  and  four 
statas  of  tha  hypothasis  var labia  (Figura  D-4) . It  la  a 
solid  which  satisfies  tha  following  conditions  which  ara 
raquirad  by  tha  rulas  of  probabilitiast 

p(l)  ♦ p(2)  + p ( 3 ) ♦ p (4)  - 1.0 

0.0  < p(l)  < 1.0 

0.0  < p(2)  < 1.0 

0.0  < p ( 3)  < 1.0 

0.0  < p (4)  < 1.0. 


STATE 

3 


STATE 

1 


Figure  D-4 

PROBABILITY  TETRAHEDRON 


Tha  rapresantation  of  probability  points  in  tha  proba- 
bility tatrahadron  is  dafinad  in  axactly  tha  same  mannar  as 
tha  probability  triangla.  In  tha  probability  triangla  tha 
probability  of  any  point  insida  tha  triangla  was  dafinad  as 
tha  distanca  of  tha  point  from  aach  of  tha  thraa  sidas;  in 
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the  probability  tetrahedron  the  probability  of  any  point  in 
tha  volume  is  defined  as  the  distance  of  the  point  from  each 
of  the  four  faces. 

One  way  to  describe  the  position  of  the  probability  point 
(bug)  is  in  terms  of  four  sets  of  parallel  planes,  each  set 
parallel  to  one  of  the  faces  of  the  tetrahedron.  For 
example,  one  set  of  planes  is  parallel  to  the  face  of  the 
tetrahedron  which  is  opposite  vertex  1 (Figure  D-5) . Each 
of  these  planes  represents  a different,  but  constant,  proba- 
bility that  state  1 will  occur.  The  plane  which  is  located 
a tenth  of  the  way  from  the  face  to  the  vertex  represents  a 
probability  of  state  1 which  is  equal  to  0.1,  the  one  which 
is  halfway  represents  a probability  of  0.5,  and  so  on. 


STATE 

3 


Figure  D-5 

PROBABILITY  TETRAHEDRON  WITH  PROBABILITY  PLANES 


In  exactly  the  same  way  there  are  planes  parallel  to 
the  face  opposite  vertex  2 which  represent  various  prob- 
bilities  that  state  2 will  occur  (Figure  D-6) , and  there  are 
planes  parallel  to  the  two  remaining  faces  which  represent 
the  probabilities  of  states  3 and  4 (Figures  D-7  and  D-8) . 
Any  particular  point  is  thus  defined  as  the  intersection  of 
four  planes,  one  in  each  of  four  directions,  which  pass 
through  it. 
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- Figure  D-6 

PROBABILITY  TETRAHEDRON  WITH  PROBABILITY  PLANES 
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STATE 


t 

Figure  D-7 

PROBABILITY  TETRAHEDRON  WITH  PROBABILITY  PLANES 
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Figure  D-8 

PROBABILITY  TETRAHEDRON  WITH  PROBABILITY  PLANES 
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For  example , the  point  representing  the  probabilities 
p(l)  - 0.1,  p (2)  » 0.2,  p (3)  «0.3,  p (4)  » 0.4  is  defined  by 
a plane  parallel  to  the  face  of  the  tetrahedron  opposite 
vertex  1 and  one  tenth  of  the  way  between  the  face  and  the 
vertex  (Figure  D-9) , a plane  parallel  to  the  face  of  the 
tetrahedron  opposite  the  vertex  marked  2 and  one-fifth  of 
the  way  from  the  face  to  the  vertex  (Figure  D-10) , a plane 
parallel  to  the  bottom  of  the  tetrahedron  and  three- tenths 
of  the  way  from  the  bottom  to  the  top  (Figure  D-ll)  and  a 
final  plane  parallel  to  the  face  of  the  tetrahedron  opposite 
vertex  4 and  four-tenths  of  the  way  from  the  face  to  the 
vertex  (Figure  D-12) . 
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Figure  D-9 

PROBABILITY  TETRAHEDRON  SHOWING 
PROBABILITY  BUG  DEVELOPMENT 


Figure  D-10 

PROBABILITY  TETRAHEDRON 
SHOWING  PROBABILITY  BUG  DEVELOPMENT 
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Figure  D-1 1 

PROBABILITY  TETRAHEDRON  SHOWING 
PROBABILITY  BUG  DEVELOPMENT 
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Figure  D-12 

PROBABILITY  TETRAHEDRON  SHOWING 
PROBABILITY  BUG  DEVELOPMENT 
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Another  way  to  describe  the  position  of  the  probability 
bug  in  space  is  to  show  the  distance  of  the  bu?  from  each  of 
the  four  faces.  In  order  to  get  a better  view  of  the  bug, 
imagine  that  the  tetrahedron  is  turned  around  on  its  vertical 
axis  to  the  right,  so  that  vertex  4 points  out  of  the  page, 
vertex  1 is  to  the  right,  and  vertex  2 points  toward  the 
left.  The  probability  of  state  1 is  represented  by  the 
distance  of  the  probability  bug  from  the  face  opposite 
vertex  1,  shown  as  an  arrow  in  Figure  D-13) . 
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Figure  D-13 

LOCATION  OF  PROBABILITY  BUG,  STATE  1 


Similarly,  the  probability  of  state  2 is  represented  by 
the  distance  of  the  probability  bug  from  the  face  opposite 
vertex  2 (Figure  D-14) , the  probability  of  state  three  by 
the  distance  from  the  bottom  face  (Figure  D-15)  and  the 
probability  of  state  4 is  represented  by  the  distance  to  the 
face  in  the  back  which  is  opposite  vertex  4 (Figure  D-16) . 
The  location  of  the  probability  bug  is  thus  represented 
completely  by  the  lengths  of  the  four  arrows  (Figure  D-17) . 
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Figure  D-16 

LOCATION  OF  PROBABILITY  BUG,  STATE  4 


STATE 


Figure  D-17 

LOCATION  OF  PROBABILITY  BUG 
STATES  1 THROUGH  4 


D.  5 Projection  Method 


The  projection  display  is  illustrated  for  the  case  of 
four  states  and  three  actions  as  shown  in  Figure  D-18.  The 
tetrahedron  is  divided  into  three  action  volumes  by  threshold 
planes.  The  plane  separating  actions  A and  B passes  through 
the  points: 


p(l) 

» .5,  p (2)  » 

.5, 

P (3) 

■ 0,  p (4)  - 0; 

p(l) 

» 0,  p (2)  ■ 

.67, 

p (3) 

■ .33,  p(4)  ■ 0;  and 

p(l) 

- 0,  p (2)  - 

.75, 

p(3) 

» 0,  p (4)  - .25. 

The  threshold  plane  between  actions  A and  C passes  through 
the  points: 

P(l)  * .4,  p (2)  « 0,  p (3)  * 0,  p(4)  - .6; 
p (1)  » 0,  p (2)  * .4,  p (3)  = 0,  p (4)  - .6;  and 
p (1)  - 0,  p (20  * 0,  p (3)  * .4,  p (4)  » .6. 

Thus,  for  the  probability  point,  p(l)  ■ .1,  p(2)  * .2,  p ( 3) 

* .3,  p ( 4 ) = .4,  action  A is  preferred,  and  for  the  prob- 
ability point,  p(l)  -.05,  p (2)  = .05,  p ( 3)  = .2,  p(4)  -.7, 
action  C is  preferred. 

The  projection  method  consists  of  displaying  the  projection 
of  the  thresholds  and  the  probability  bug  onto  faces  of  the 
tetrahedron.  Two  of  the  four  possible  projections,  on  the 
1-2-3  and  2-3-4  faces,  are  shown  at  the  bottom  of  Figure  D-18. 
The  projections  present  the  inside  of  the  tetrahedron  as  it 
would  be  seen  viewed  perpendicularly  through  these  faces. 

The  main  difficulty  with  the  projection  method  is  that 
the  thresholds  are  not  probability  lines,  as  they  are  in  the 
triangular  representation  of  three  states,  they  are  probability 
bands.  This  feature  sometimes  makes  it  difficult  or  impossible 
to  tell  exactly  where  the  probability  bug  is  and,  hence, 
what  the  preferred  decision  is.  For  instance,  consider  the 
1-2-3  projection  on  the  bottom  left  of  Figure  D-18.  In  this 
projection,  the  threshold  between  Actions  A and  C appears  as 
a large  triangle  in  the  center  of  the  probability  triangle. 
Furthermore,  whenever  the  probability  bug  is  located  within 
the  threshold  band,  it  will  appear  the  same,  regardless  of 
which  action,  A or  C,  is  preferred.  Thus  for  the  probabilities 
shown,  the  action  indicator  appears  the  same  despite  the 
fact  that  action  A is  preferred  at  the  dot,  , and  action 
C is  preferred  at  the  "X."  However,  other  views  may  not 
have  this  problem,  as  shown  in  the  2-3-4  projection.  Here 
it  is  quite  clear  that  action  C is  preferred  at  the  "X,”  and 


action  A at  tha  "dot."  It  remains  to  ba  saan  whathar  this 
ambiguity  prasants  a problem  whan  actual  daciaions  ara 
modalad.  For  example,  it  may  ba  posaibla  to  display  tha 
path  of  tha  probability  bug  as  a dashad  lina  whenever  it 
lias  "behind"  tha  threshold  plana.  In  any  case#  it  is  clear 
that  use  of  this  method  requires  careful  consideration  of 
tha  face  to  ba  used  for  projection. 

D. 6 Cutting-Plane  Method 

Tha  essence  of  tha  cutting-plane  method  is  illustrated 
in  Figure  d-19,  again  using  tha  probability  tetrahedron.  As 
noted  in  Section  3.1.3,  tha  tetrahedron  may  ba  described 
using  planes  parallel  to  tha  faces  which  pass  through  tha 
probability  point.  These  planes  form  the  cutting-plane 
representation  at  the  given  probability  point.  For  example, 
through  the  point  p(l)  ».l,  p(2)  » .2,  p(3)  • .3,  and  p(4)  • 
.4,  the  cutting  plane  parallel  to  the  1-2-3  face  has  the 
equation:  p(4)a.4  and  the  cutting  plane  parallel  to  the  2-3- 
4 face  has  the  equation  p(l)«.l.  These  cutting  planes  are 
displayed  at  in  the  bottom  of  Figure  D-19. 

Next,  let  us  investigate  the  threshold  representation 
in  the  cutting  planes.  Consider  the  four-state  value  table 
shown  in  Table  D-l,  which  can  be  assessed  in  the  manner 
explained  in  pages  3-6  through  3-9  of  Brown,  et  al.  (1975) . 
From  this  value  table,  the  following  thresholds  can  be 
calculated^-: 


A/B  Threshold:  p{2)  • p(l)  ♦ p(3)  + 2.8p(4) 

p(l)  < .5 
p ( 3 ) < .333 
p (4)  < .263 


A/C  Threshold: 


p (4)  » 1.7p(l)  + 1. 7p (2)  + 1. 7p (3) 
p(l)  < .37 
p (2)  < .37 
p ( 3)  * .37 


The  display  of  the  thresholds  in  the  tetrahedron  is 
shown  at  the  top  of  Figure  D-20,  and  the  intersection  of  the 
thresholds  with  the  cutting  planes  is  shown  at  the  bottom  of 
Figure  D-20.  Thus,  in  contrast  with  the  projection  method, 
the  thresholds  in  the  cutting  planes  consist  of  lines,  and 
there  is  no  ambiguity  over  the  position  of  the  probability 
bug  with  respect  to  the  thresholds  provided  that  the  thresholds 
appear  in  the  cutting  plane.  Notice  however,  that  the 
threshold  lines  appear  in  the  p(l)  - .1  cutting-plane,  but 


1 These  thresholds  are  approximately  the  same  as  those  used 
in  the  illustration  of  the  projection  method.  Figure  D-18. 
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CUTTING-PLANE  METHOD 
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not  in  the  p (4)  - .4  cutting  piano.  Thia  is  because,  with 
p(4)  - .4/  ths  probability  of  p(l),  p(2),  or  p(3)  could  not 
change  enough  to  cause  another  action  to  be  preferred.  This 
means  that  some  cutting  planes  may  display  essentially  no 
information  about  thresholds,  and  this  may  not  be  useful  to 
the  user.  For  this  reason  the  choice  of  cutting  planes  is 
very  important,  as  is  the  choice  of  the  projection  face  in 
the  projection  method. 

The  thresholds  within  the  cutting  planes  are  calculated 
from  the  data  in  the  value  table  and  from  the  location  of 
the  probability  bug.  For  example,  the  threshold  equations 
for  the  cutting  plane  parallel  to  the  2-3-4  face  are  obtained 
by  substituting  p(l)  » .1  in  the  threshold  equations  of 
Table  0-1.  Figure  D-21,  at  the  top  left,  shows  a convenient 
format  for  this  calculation.  First,  copy  the  last  three 
columns  of  the  value  table,  those  relating  the  actions.  A, 

B,  C,  to  states  2,  3,  4.  Next,  to  each  row,  add  p(l)  * .1 
times  the  value  for  the  row  shown  in  the  state  1 column  in 
Table  0-1  (for  example,  add  .1  x (-25)  * -2.5  to  each  entry 
in  action  B row) . This  calculation  yields  the  value  table 
at  the  top  right  of  Figure  D-21,  which  can  be  used  to  derive 
the  threshold  equation  illustrated  and  plotted  in  the  lower 
part  of  the  figure.  The  convenience  of  this  procedure  is 
seen  when  the  position  of  the  probability  bug  changes,  for 
instance,  to  p(l)  ■ .05,  p(2)  ■ .05,  p(3)  * .2,  p(4)  ■ .7. 

As  shown  in  Figure  D-22,  this  change  is  accommodated  by 
changing  the  amount  added  to  each  row  due  to  column  l's 
value . 


This  probability  change  is  also  illustrated  in  Figure  D-23. 

As  shown  in  the  bottom  figure,  changing  only  the  probability 
changes  both  the  bug  position  and  the  thresholds.  In  the 
case  of  the  cutting  plane  parallel  to  the  1-2-3  face  this 
probability  change  caused  the  entire  display  to  change  from 
favoring  action  A to  favoring  action  C.  The  sizes  of  the 
cutting  planes  have  also  changed,  reflecting  the  change  in 
the  degrees  of  freedom  for  the  remaining  probabilities. 

Thus,  the  revised  cutting  plane  parallel  to  the  1-2-3  face 
allows  the  three  probabilities  to  vary  between  0 and  .30, 
whereas  they  were  formerly  free  to  vary  from  0 to  .60.  This 
represents  a reduction  of  one-half,  and  the  triangle  in 
Figure  0-23  is  half  as  big  as  it  was  in  Figure  D-20. 

The  cutting-plane  method  appears  to  have  several 
attractive  properties.  It  does  not  force  the  model  to  take 
a constrained  form,  it  communicates  the  dynamic  situation 
well,  and  it  presents  unambiguous  threshold  lines.  The 
major  disadvantage  is  that  any  the  movement  of  the  proba- 
bility bug  causes  the  probability  location,  the  threshold 
locations  on  the  cutting  planes,  and  the  size  of  the  tri- 
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-25p (2)  - 30p  (4)  - -2 . 5p (2)  - 52 . 5p (3)  - 102.5p(4) 
p (2)  - 2 . 3p  (3)  ♦ 3 . 2p  (4 ) 

Threshold 

-25p (2)  - 30p (4)  ■ -80p (2)  - 55p(3)  - 5p(4) 
p (4)  - 2 . 2p  ( 2)  ♦ 2 . 2p  (3) 


P (3)  * .9 


Figure  D-21 

INITIALTHRESHOLD  CALCULATION 
FOR  THE  2-34  CUTTING  PLANE 


L2 
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angles  to  change.  This  Is  in  contrast  to  the  original 
Execution  Aid,  in  which  movement  in  the  probability  bug 
caused  only  the  probability  location  to  change  (the  thresh- 
old locations  and  the  size  of  the  display  remained  fixed) . 
The  changing  threshold  locations  are  a problem  because  they 
allow  for  the  possibility  of  surprises.  That  is,  the  proba- 
bility bug  may  cross  a threshold  that  was  not  previously, 
displayed,  causing  a new  action  to  be  preferred.  Further, 
the  changing  sizes  of  the  displays  is  a potential  problem 
because  it  may  have  a somewhat  disconcerting  effect  on  a 
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